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INTRODUCTION 


The  purpose  of  this  project  is  to  refine  a  “visual  neural  network,”  combined  with  a 
mammographic  database,  and  test  it  for  its  ability  to  help  radiologists  reduce  the  number  of 
benign  biopsies  in  mammographic  screening  without  increasing  the  number  of  missed  cancers. 
We  refer  to  this  as  a  “mapped  database  diagnostic  system.”  The  unique  features  of  this  system 
are  a)  its  exploitation  of  a  clinically  proved  database  of  mammograms  for  enhanced  diagnosis,  b) 
automatic  selection  of  highly  discriminating  mammographic  features,  and  c)  a  two-dimensional 
“relational  map”  for  enhanced  browsing  through  the  mammographic  database. 

Years  1  and  2  of  this  project  will  be  devoted  primarily  to  retrospective  studies.  Years  3  and  4 
primarily  to  clinical  studies. 


BODY 

Task  1.  Devise,  construct,  and  test  an  initial  lesion  detector  for  mammography  (months  1-12). 

a.  Modify  earlier  software  for  detection  of  microcalcifications  to  admit  additional  types 
of  lesions,  including  masses,  stellate  lesions,  asymmetries  and  architectural 
abnormalities,  (months  1-2) 

b.  Construct  new  algorithms  for  mass  detection  based  on  known  techniques,  such  as 
Gaussian  derivative  operators,  difference  of  Gaussians,  Laws  texture  features, 
density-weighted  contrast  enhancement,  and  Laplacian-Gaussian  edge  detection. 
Incorporate  this  algorithm  in  the  algorithm  for  microcalcifications  detection.  Test 
algorithm  on  digital  image  data  obtained  in  Task  3b.  (months  1-6) 

c.  Construct  new  algorithms  for  detection  of  asymmetries  and  architectural  distortions. 
This  detector  may  be  based  on  recent  work  on  model-based  analysis  of 
mammograms.  Incorporate  this  detector  in  the  algorithm  for  microcalcification  and 
mass  detection.  Test  algorithm  on  image  data  obtained  in  Task  3h.  (months  7-12) 

In  accordance  with  Task  la  and  Task  lb,  we  developed  a  lesion  detector  for  masses,  and  tested  it 
successfully  on  a  test  set  of  regions  of  interest  from  the  University  of  South  Florida  (USF) 
database.  Task  Ic  will  be  postponed  to  Year  2  or  Year  3. 

Task  2.  Construct  advanced  lesion  detector  (months  4-9). 

a.  The  panel  of  radiological  fellows  will  label  each  of  the  candidate  lesions  produced  by  the 

initial  lesion  detector  as  mass,  microcalcifications,  asymmetry,  architectural 
distortion,  or  nonlesion,  (months  4-5) 

b.  Build  an  algorithm  for  extracting  about  40  morphological  features  from  the  candidate 

lesions  based  on  earlier  work  [30].  (months  4-7) 

c.  Build  a  genetic  algorithm  for  selecting  no  more  than  10  morphological  features  for  use  in 

a  lesion  classifier.  The  error  rate  of  a  A:-nearest-neighbor  classifier  will  be  the  penalty 
function  for  this  genetic  algorithm  in  accordance  with  [25].  The  same  genetic  process 
that  selects  the  features  will  choose  the  value  of  k.  (months  8-9) 

d.  Combine  the  ^^-nearest-neighbor  classifier  with  the  initial  lesion  detector  to  form  an 

advanced  lesion  detector,  (month  10) 
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The  “advanced  lesion  detector”  in  Task  2d  was  combined  with  the  neural  classifier  of  Task  5, 
described  below.  Tasks  5a  and  5b  were  implemented  this  year,  five  months  ahead  of  schedule. 
We  plan  to  implement  Task  5c  in  month  17,  as  originally  scheduled. 

Task  3.  Collect  and  organize  a  database  of  proven  mammographic  regions  of  interest  (months 
1-6, 12-24)  from  at  least  400  subjects,  restricting  the  lesions  to  microcalcifications  and  masses. 

a.  Collect  film  mammograms  from  at  least  400  subjects,  restricting  the  lesions  to 
microcalcifications  and  masses.  (The  masses  will  include  stellate  lesions.)  Add 
teaching  files  if  possible.  Include  films  from  King-Drew  Medical  Center  (KDMC) 
and  from  University  of  California  at  Los  Angeles  (UCLA),  (months  1-2) 

b.  Digitize  all  films  on  Lumiscan  85  film  scanner  at  a  50-micron  pixel  width.  Save  all 
digitized  mammograms  on  compact  disc,  (month  2) 

c.  On  each  mammogram  find  one  or  more  lesions,  using  the  advanced  lesion  detector  of 
Task  2.  Enclose  one  or  more  of  these  lesions  in  a  512x512-pixel  or  a  1024x1024  - 
pixel  square.  We  refer  to  this  square  as  a  region  of  interest  (ROI).  (months  3-4) 

d.  Print  hard  copies  of  all  of  the  digitized  ROIs  collected  in  Task  3c.  Each  radiologist 
on  this  project  will  partition  the  ROIs  into  groups  of  medically  and  perceptually 
similar  ROIs.  A  medically  or  visually  descriptive  label  will  identify  each  group.  We 
refer  to  these  as  perceptual  similarity  groups.  The  radiologists  will  form  a  consensus 
on  a  final  grouping  and  labeling  of  these  groups.  (Month  5) 

e.  Each  radiologist  will  construct  a  dissimilarity  matrix  for  these  groups.  Each  element 
of  this  matrix  will  contain  the  radiologist’s  subjective  estimate  of  the  dissimilarity  of 
the  corresponding  two  groups  on  a  scale  from  0  to  5, 0  designating  identity,  and  5 
designating  extreme  dissimilarity.  (Month  5) 

f  Label  the  resulting  ROI  as  one  of  {mass,  calcification,  asymmetry,  architectural 
distortion}  and  as  one  of  {benign,  malignant}.  (Month  5) 

g.  Partition  the  database  of  ROIs  into  D  and  T,  in  a  manner  so  that  the  distributions  of 
types  of  abnormalities  and  perceptual  similarity  groups  in  D  and  Tare  similar. 

(Month  6) 

We  collected  and  organized  a  database  of  fiill-field  mammograms  and  proven  regions  of  interest 
(ROIs).  This  database  consisted  of  about  1000  cases  from  the  University  of  South  Florida 
(USF),  160  cases  from  the  King-Drew  Medical  Center  (KDMC),  and  10  cases  from  the  Iris 
Cantor  mammography  center  at  the  University  of  California  at  Los  Angeles  (UCLA).  The  cases 
were  described  on  a  report  form  designed  jointly  by  members  of  this  project  at  KDMC  and 
UCLA.  A  copy  of  this  form  is  provided  in  Appendix  1.  The  radiology  fellows  in  our  project 
circumscribed  each  observed  abnormal  region  by  a  rectangle  aligned  with  the  horizontal  and 
vertical  axes.  Thus  we  completed  Tasks  3a,  3b,  and  3c.  Tasks  3d,  3e,  3f,  and  3g  have  been 
postponed  to  Year  2. 

Task  4.  Construct  a  graphical  user  interface  for  interacting  with  the  two-monitor  high-resolution 
display  and  the  control  monitor.  Include  a  high-resolution  simulated  square  magnifying  glass 
that  shows  a  window  excised  from  the  4000  x  5000-pixel  array  produced  by  the  scanner. 

Include  a  brightness  inversion  capability.  Include  a  brightness  control  and  a  contrast  control, 
(months  14-15).  This  Task  was  completed  3  months  ahead  of  schedule. 
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Task  5.  Construct  and  train  a  visual  neural  classifier  and  a  relational  map.  (months  16-17) 

a.  Extract  the  set  of  features  F2  from  the  design  set  D.  Use  the  genetic  algorithm  G2  to 
find  a  set  of  reduced-dimensionality  features  for  analyzing  the  ROIs.  Using  these 
reduced-dimensionality  features  for  design  set  D,  construct  a  database  K  of  labeled 
reduced-dimensionality  feature  vectors  for  training  the  visual  neural  classifier. 

(month  16) 

b.  Construct  an  initial  5-layer  visual  neural  network  and  a  relational  map,  based  on  the 
design  principles  described  in  [18-20].  (month  17) 

c.  Train  this  neural  network  using  the  design  set  K  to  preserve  perceptual  similarity 
grouping  and  benign-malignant  separation  in  the  relational  map.  (month  17) 

Tasks  5a  and  5b  were  completed  ahead  of  schedule.  The  fimction  of  the  5-layer  neural  network 
in  Task  5b  was  expanded  to  include  the  “advanced  lesion  detector”  of  Task  2d. 

Task  6.  Construct  a  mapped  database  system  (originally  called  “neurodatabase  system,”) 

consisting  of  a  graphical  user  interface,  a  relational  map,  the  visual  neural  classifier, 
and  the  proven  mammographic  database,  (month  17) 

This  Task  was  completed  successfully,  5  months  ahead  of  schedule. 

ADDITIONAL  ACTIVITIES: 

1.  We  carried  out  an  ROC  analysis  of  the  4  unaided  radiologists,  the  4  aided  radiologists, 
and  the  computer  alone  for  our  earlier  retrospective  test  of  our  mapped  database 
diagnostic  system.  The  results  of  this  test  show  that  radiologists  may  achieve 
statistically  significantly  reductions  in  the  numbers  of  benign  biopsies  and  misdiagnosed 
cancers  with  the  aid  of  a  mapped  database  of  proved  mammographic  ROIs.  This  test  is 
described  in  a  paper  published  in  the  June  2000  issue  of  Academic  Radiology.  A  cop  y 
of  this  paper  is  given  in  Appendix  2. 

2.  We  devised  and  implemented  a  novel  computer  architecture  for  our  mapped  database 
diagnostic  system  that  will  facilitate  incorporating  this  system  in  multisatellite 
teleradiology  systems.  This  architecture  includes  a  Java-expressed  user  interface  that 
can  be  tied  to  a  wide  variety  of  remotely  sited  workstations,  each  with  its  distinct 
operating  system.  We  plan  to  present  this  architectural  concept  at  the  2001  SPIE 
International  Symposium  on  Medical  Imaging  in  San  Diego. 

RELEVANCE  OF  THE  ABOVE  WORK  TO  THE  ORIGINAL  HYPOTHESIS. 

The  following  is  our  original  hypothesis:  Compared  to  an  maided  radiologist  a  neurodatabase 
diagnostic  system  assisting  a  radiologist  in  mammographic  screening  can  significantly  reduce 
both  the  incidence  of  negative  biopsies  as  well  as  the  incidence  of  missed  cancers. 

The  published  paper  (given  in  Appendix  2)  shows  the  validity  of  this  hypothesis  when  the 
image  data  are  restricted  to  regions  of  interest  containing  either  microcalcifications  or  normal 
tissue. 


7 


KEY  RESEARCH  ACCOMPLISHMENTS 


•  Constuction  of  relational  map  for  mammographic  masses 

•  Determination  of  discriminating  features  for  mass  detection 

•  Construction  of  mass  detector 

•  Testing  of  mass  detector 

•  Demonstration  of  statistical  improvement  of  database-aided  reader  over  unaided  reader 
with  respect  to  both  misdiagnosed  breast  cancers  and  falsely  detected  breast  cancers. 

•  Design  and  construction  of  platform-independent  architecture  for  multisatellite  remotely 
aided  diagnosis  of  mammograms. 


REPORTABLE  OUTCOMES 
Manuscripts,  abstracts,  articles,  presentations: 

J.  Sklansky,  Eric  Y.  Tao,  M.  Bazargan,  C.J.  Omes,  R.C.  Murchison,  S.  Teklehaimanot,  “Computer- 
Aided,  Case-Based  Diagnosis  of  Mammographic  Regions  of  Interest  Containing 
Microcalcifications,”  .<4cac/e/n/c  Radiology,  Vol.  7,  No.  6,  June  2000,  pp.  395-405. 

Y.  Price,  J.  Sklansky,  E.Y.  Tao,  C.J.  Omes,  M.  Bazargan,  S.  Teklehaimanot,  “Mapped  Database  for 
Case-Based  Diagnosis  in  Digital  Mammography,”  Scientific  Exhibit  on  Breast  Imaging  at  1999 
Annual  Meeting  of  the  Radiological  Society  of  North  America.,  Radiology,  Vol.  213(P),  November 
1999,  page  505. 

J.  Sklansky,  E.  Y.  Tao,  C.  Omes,  A.  C.  Disher,  “A  Visualized  Mammographic  Database  in 
Computer-Aided  Diagnosis,”  Computer-Aided  Diagnosis  in  Medical  Imaging,  edited  by  K.  Doi, 
H.  McMahon,  M.  L.  Giger,  K.R.  Hoffmann,  Elsevier,  Amsterdam,  1999,  pp.  215-220. 

Jack  Sklansky,  EngScD,  Chester  J.  Omes,  MS,  “Database-Aided  Diagnosis  in  Digital 
Mammography,"  wybRAD  Exhibit,  2000  Annual  Meeting  of  the  Radiological  Society  of  North 
America,  November  2000  (in  preparation). 

Funding  based  on  work  supported  by  this  project: 

“Database-Aided  Telemammography,”  grant  of  $100,000  from  California  Telehealth  and 
Telemedicine  Center,  for  the  period  April  1,2000  to  March  31, 2002.  Project  Director:  Jack 
Sklansky,  EngScD. 

Employment  or  research  opportunities  applied  for  and/or  received  on  experiences/training 
supported  by  this  award: 

Famoosh  Nooiyanni,  MD,  received  and  accepted  an  offer  of  a  residency  in  Radiology  at  the 
University  of  Southern  California  with  the  help  of  her  experience  as  a  Radiology  Fellow  on  our 
project.  This  appointment  will  begin  in  July  200 1 . 
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Yvette  Price,  MD,  received  and  accepted  an  appointment  as  a  Radiology  Fellow  at  the  UCLA  Iris 
Cantor  Center  for  Breast  Imaging,  beginning  in  July  2001.  Dr.  Price  was  a  Radiology  Resident  and 
participated  in  our  project’s  research  at  the  time  that  she  applied  for  this  Fellowship. 


CONCLUSIONS 

Our  diagnostic  system  shows  promise  of  several  important  advances  in  the  field  of 
computer-aided  diagnosis  (CAD).  One  of  these  advances  is  the  automation  of  the  selection 
of  groups  of  image  features  that  are  good  discriminators  between  malignant  and  benign 
lesions.  Another  advance  is  the  demonstration  in  our  recent  study  (Appendix  2)  that  a 
properly  designed  interface  between  a  human  reader  and  a  proved  database  of 
mammographic  images  may  enable  less  experienced  readers  to  exceed  the  performances  of 
unaided,  highly  skilled  readers  -  without  the  need  for  time-consuming  training  oon  feature 
extraction. 

Our  design  of  a  platform-independent  architecture  for  multisatellite  remotely  sited  diagnosis 
of  mammograms  shows  promise  of  accelerating  the  development  and  growth  of  multi¬ 
satellite  teleradiology,  because  it  permits  the  use  of  low-cost  PC  computers  in  conjunction 
with  state-of-the-art  high-resolution  radiological  monitors. 
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Appendix  I 


Mammography  Report 


Study  Identification  Number  Age 

Screening  Assessment  Oa  Ou  Oo  1  2  3  4  5  Date 

Final  Assessment  1  2  3  4a  4b  4c  5  Date 

Br^stComposlison  1  2  a  4  '  — ~ 

Clinical  Correlation  Finding  correlates  to  clincial  exam  finding  in  QL  OR  QB  breast(s)at. 


Oa  =  Additional  films  1  s=  Almost  ©ntirely  fat 

Ou  =  Ultrasound  2  =  Scattered  fibrDgl£indular  densities 

Oo  =  Outside  Films  3  =  Heterogeneously  dense  I 

1  =  Negative  4  =  Extremely  dense  L, 

2  =  Benign  finding 

3  =  Probably  benign 

4  =  Suspicious  abnormality  (a-mild,  b-moderate,  c-high) 

5  =  Highly  suggestive  of  malignancy 


V“  f  jV 


MASS 

In  area 

Size  /Number 

of  density? 

Shape 

Margins 

Density 

_ cm  _ # 

□  Yes 

□  No 

□  Round  OljobuUir 

□  Oval  □  Irregular 

□  Cfrctimscribed  □  Ill-defined 

□  MIcrolobulated  □  Spiculated 

□  Obscured 

□  High 

□  Low 

□  Equal 

□  Fat  containing 

_ _  cm  _ # 

□  Yes 

□  No 

□  Round  □Lobular 

□  Oval  □  Irregular 

□  CircunmcHbed  □  Ill-defined 

□  MIcrolobulated  □  Spiculated 

□  Obscured 

□  High 

□  Low 

□  Equal 

□  Fat  containing 

CALCinCATIONS 

Distribution  Modifiers 

□  Grouped/Clustered 

□  Linear 

□  Segmental 

□  Regional 

□  Diffuse/Scattered 


Other  Findings 

□  Skin  thickening 

□  Trabecular  thicUening 

□  Skin  lesion 


□  Skin 

□  Vascular 

□  Coarse 

□  Large  rod-like 

□  Round 


□  Lucent-centered 

□  Skin  retraction 

□  Nipple  retraction 

□  Axillary  adenopathy 


□  Eggshell/Rim 

□  Milk  of  Calcium 

□  Suture 

□  Dystrophic 

□  Punctate 


Intermediate  Concern 
□  Amorphous  or  Indistinct 


Q  Pleomorphic/hetereogenous 
□  Fine,  linear  branching 


Q  Asymmetric  breast  tissue 
□  Focal  asymmetric  density 
Q  Architectural  distortion 


p4..wology  Diagnosis; 
Comments: 


Benign 


Malignant 


PATHOLOGY  CODES 

PATHOLOGY  •  MALIGNANT 
i,NC  MetatUUc  difea«e  Axillary  Node 

IC  C  ollold  (muclBOM)  C  arclnoma 
:i  C  omedocarelnoma  d  BtradacUl) 

;p  'ntra  C  ystlc  P  apilloma 
»S  octal  eardnoma  In-S  Ito  DCIS 

C  I  otracyattc  papUlary  C  ardnoma 
D  I  OTadre  D  octal  cardooma 

^  I  oraftTc  aod  I  o«rito  caocer 

L  I  OTadre  L  obolar  ca^ooma 

N  IN  flamoiatory  cardnooia  - - 

*  P  P  apillary  I  oradre  .  cardnooia 
,N  L  ymph  N  ode 
Y  LY  mphoma 
IB  M  etastadc  ledon  to  the  B 
fC  M  edullary  C  arclnoma  ’ 

II  M  oltifocal  I  ntradnctal  eardnoma 
C  P  apillary  C  ardnoma  In-dto  . 

D  P  aget'a  D  Icease 
.M  R  ecorrent  M  allgnancy 
S  arcoma 

C  T  obolar  C  arclooma 

•THER: _ 

PATHOLOGY  •  HIGH  RISK  LESIONS 
TD  AT  ypleal-D  octal  byperplaala 
TL  AT  yplcai  L  obolar  hyperplasia 
S  obolar  Cardooma  In  S  ito 

p  P  erlpheral:  doct  P  apilloma 


PATHOLOGY  -  BENIGN 


AB 
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AD 
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Computer-aided,  Case-based  Diagnosis 
of  Mammographic  Regions  of  Interest 
Containing  Microcaicifications' 

Jack  Sklansky,  EngScD,  Eric  Y.  Tao,  PhD,  Mohsen  Bazargan,  PhD 
Chester  J.  Ornes,  MS,  Robert  C.  Murchison,  MD,  Senait  Teklehaimanot,  MPH 


Rationale  and  Objectives.  The  purpose  of  this  study  was  to  evaluate  the  effectiveness  of  a  mapped-database  diagnostic  sys¬ 
tem  in  reducing  the  incidence  of  benign  biopsies  and  misdiagnosed  cancers  among  mammographic  regions  of  interest  (ROIs). 

Materials  and  Methods.  A  novel  neural  network  was  devised  (a)  to  respond  to  a  query  ROI  by  recommending  to  biopsy  or 
not  to  biopsy  and  (b)  to  map  each  ROI  in  the  database  as  a  dot  on  a  computer  screen.  The  network  was  designed  so  that  clus¬ 
ters  in  the  array  of  dots  help  the  radiologist  to  find  proved  ROIs  visually  similar  to  the  query  ROI.  This  mapped-database  diag¬ 
nostic  system  was  restricted  to  ROIs  with  visible  microcalcifications.  The  neural  network  was  trained  with  a  stored  database  of 
80  biopsy-proved  ROIs. 

Results.  Four  radiologists  acting  independently  on  100  ROIs  recommended  biopsies  for  18, 15,  28,  and  18  benign  ROIs  and 
misdiagnosed  cancers  in  11,  12,  7,  and  eight  ROIs,  respectively.  Interaction  with  the  mapped-database  system  reduced  the 
numbers  of  benign  biopsies  to  11,  eight,  18,  and  10  cases  and  of  misdiagnosed  cancers  to  eight,  seven,  four,  and  three  cases, 
respectively.  Statistical  analysis  indicated  that  three  radiologists  achieved  significant  improvements  at  P  <  .02  and  the  fourth 
achieved  a  substantial  improvement  at  P  <  .07. 

Conclusion.  By  using  a  mapped  database  of  proved  mammographic  ROIs  containing  microcalcifications,  radiologists  may 
statistically  significantly  reduce  the  numbers  of  benign  biopsies  and  misdiagnosed  cancers. 

Key  Words,  Computer-aided  diagnosis;  digital  mammography;  microcalcifications. 


A  critical  aspect  of  mammographic  diagnosis  is  deciding 
whether  to  recommend  biopsy.  Approximately  two  benign 
lesions  are  sampled  for  biopsy  for  every  malignant  lesion 
detected  (1).  In  a  recent  study  of  2,400  women  undergoing 
mammographic  screening  during  a  10-year  period  (2),  an 
additional  $33  was  spent  on  evaluating  false-positive  re- 
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suits  for  every  $100  spent  on  screening.  Among  the  women 
in  that  study,  the  cumulative  risk  of  a  false-positive  result 
after  10  mammograms  was  49.1%.  Thus,  it  would  be  highly 
desirable  to  reduce  the  frequency  of  benign  biopsies  during 
mammographic  screening  without  increasing  the  number  of 
missed  cancers. 

The  objective  of  this  study  was  to  evaluate  the  effective¬ 
ness  of  a  recently  devised  “mapped-database  diagnostic 
system”  in  reducing  the  frequency  of  benign  biopsies  and 
misdiagnosed  cancers  among  mammographic  regions  of 
interest  (ROIs).  The  principal  difference  between  this  sys¬ 
tem  and  earlier  computer-aided  diagnostic  systems  for  mam¬ 
mography  is  the  facilitation  of  diagnostic  reasoning  by  a 
database  of  proved  ROIs.  In  this  study,  the  presented  mam¬ 
mographic  images  and  stored  database  are  restricted  to  ROIs, 
and  the  ROIs  themselves  are  restricted  to  those  revealing 
apparent  microcalcifications. 

Interpretation  of  mammograms  consists  of  two  major 
components:  detection  and  diagnosis.  In  detection,  the 
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radiologist  typically  examines  four  full-field  views — two 
views  of  each  breast.  In  these  views,  the  radiologist  may 
detect  one  or  more  ROIs  revealing  possible  lesions.  In  diag¬ 
nosis,  the  radiologist  analyzes  one  or  more  ROIs  to  deter¬ 
mine  whether  biopsy  should  be  performed  or  whether  the 
patient  should  be  recalled  for  further  examination.  Thus, 
analysis  of  ROIs  is  an  important  part  of  clinical  mammogra¬ 
phy,  particularly  in  establishing  a  diagnosis.  This  study  fo¬ 
cused  on  the  ability  of  the  mapped-database  diagnostic  sys¬ 
tem  to  reduce  the  number  of  negative  biopsy  results  and  of 
misdiagnosed  cancers  associated  with  the  analysis  of  ROIs. 

By  restricting  the  images  used  in  this  study  to  ROIs, 
thereby  excluding  the  full-breast  views,  we  reduced  much 
of  the  cost  and  duration  of  the  experiments,  but  at  the  ex¬ 
pense  of  elirhinating  contextual  evidence.  A  further  reduc¬ 
tion  in  cost— and  in  diagnostic  accuracy— was  obtained  by 
replacing  film  mammograms  with  digital  images  displayed 
with  a  cathode  ray  tube  monitor  (3).  By  anticipating  the  wide¬ 
spread  growth  of  digital  mammography  while  recognizing 
these  reductions  in  diagnostic  accuracy,  this  relatively  inex¬ 
pensive  study  examined  the  effect  of  the  mapped-database 
system  oh  diagnostic  accuracy  during  digital  mammography. 

In  the  mapped-database  system,  an  artificial  neural  net¬ 
work  responds  to  a  mammographic  ROI  by  recommending, 
or  not  recommending,  biopsy.  In  addition,  the  neural  network 
maps  a  database  of  ROIs  fix)m  proved  mammograms  onto  a 
two-dimensional  display  as  an  aid  to  establishing  a  diagnosis. 
The  network  does  this  by  mapping  each  ROI  into  a  dot  on  a 
screen  and  arranging  these  dots  so  that  they  cluster  into  radio¬ 
graphically  similar  subclasses.  We  refer  to  this  representation 
as  a  mapped  database.  In  addition  to  producing  the  mapped 
database,  the  neural  network  partitions  the  screen  into  two  de¬ 
cision  regions:  one  for  a  biopsy  being  recommended,  and  the 
other  for  a  biopsy  not  being  recommended.  We  call  the  result¬ 
ing  display  a  relational  map  (4).  By  annexing  this  neural  net¬ 
work  to  a  means  of  quickly  retrieving  and  viewing  ROIs  that 
are  mapped  close  to  the  query,  the  radiologist  is  given  a 
simple  way  of  conversing  with  the  database  to  enhance  the 
reliability  of  the  diagnostic  decision.  Thus,  this  system  ampli¬ 
fies  the  radiologist’s  powers  of  “case-based  reasoning”  (5) 
and  acts  somewhat  like  a  second  reader  with  consensus  (6). 

This  concept  led  to  construction  of  a  mammographic  di¬ 
agnostic  system  built  on  the  basis  of  several  earlier  devel¬ 
opments:  {a)  algorithms  for  detecting  and  analyzing  clus¬ 
ters  of  microcalcifications  (7-10),  {b)  large  databases  of 
digitized  mammograms  (11-15),  (c)  genetic  algorithms  for 
high-speed  parallel  search  (16),  and  (d)  “visual  neural  clas¬ 
sifiers”  (ie,  specialized  neural  networks)  that  map  multidi¬ 
mensional  features  and  multidimensional  decision  surfaces 


onto  two-dimensional  displays  (4).  We  refer  to  this  as  a 
mapped-database  diagnostic  system.  The  test  results  re¬ 
ported  herein  suggest  that  for  mammograms  with  detectable 
microcalcifications,  this  system  can  provide  a  new  level  of 
reliability  for  mammographic  diagnosis. 

The  most  pertinent  earlier  work  regarding  database-aided 
mammographic  diagnosis  is  that  by  Swett  and  his  colleagues 
at  Yale  University  (17).  Their  mammographic  database  sys¬ 
tem,  MAMMO/ICON,  required  a  verbal  medical  history  and 
verbal  mammographic  descriptors  to  supplement  each  mam¬ 
mogram.  This  system  could  provide  diagnostic  advice  that 
depended  on  semantic  similarities  of  the  verbal  descriptors 
and  medical  histories  (18).  A  shortcoming  of  this  technology, 
however,  was  its  dependence  on  verbal  descriptors  to  match 
information  that  was  essentially  pictorial  and  nonverbal. 
Among  radiologists,  there  may  be  considerable  variability  in 
the  specific  words  used  to  describe  the  pertinent  properties 
of  any  given  mammogram.  This  variability  may  impair  the 
reliability  of  retrieving  similar  maiiunograms  filed  according 
to  verbal  descriptors.  Another  shortcoming  was  the  manual 
construction  of  the  verbal  descriptors  for  the  mammograms, 
in  contrast  to  the  possibility  of  automatic  extraction  and 
counting  of  key  words  in  text. 

Among  other  published  articles  on  computer-aided  diag¬ 
nosis  of  mammograms,  those  by  Getty  et  al  (19),  Jiang  et  al 
(9),  and  Chan  et  al  (20)  are  particularly  relevant  to  the  pres¬ 
ent  work.  Getty  et  al  (19)  demonstrated  that  a  checklist  of 
descriptors  (ie,  “features”)  combined  with  a  digital  classifier 
could  be  an  effective  aid  in  establishing  the  diagnosis  during 
mammography.  These  features,  as  in  our  experiment,  were 
for  the  most  part  descriptors  of  a  “focused  abnormality”  (ie, 
the  equivalent  of  ROI  as  used  here).  The  protocol  for  aiding 
the  radiologist  included  a  computer-alone  reading,  which  was 
followed  with  a  second  reading  by  the  radiologist  (analogous 
to  a  double  reading  with  consensus).  A  second  contribution 
by  Getty  et  al  was  the  demonstration  that  a  carefully  selected 
set  of  features,  when  coupled  with  a  session  to  train  radiolo¬ 
gists  to  estimate  these  features,  could  enable  computer-aided 
generalists  to  read  film  mammograms  almost  as  well  as 
highly  skilled  specialists.  Jiang  et  al  (9)  demonstrated  the 
potential  of  automatically  extracting  features  from  microcal¬ 
cifications,  coupled  with  an  artificial  neural  network,  for 
computer-aided  diagnosis.  The  methods  in  both  of  these 
studies  (9,19),  however,  still  required  substantial  human  in¬ 
volvement  in  the  computer-aided  procedure.  In  the  method 
of  Getty  et  al  (19),  each  radiologist  estimated  the  numeric 
level  of  intensity  or  level  of  confidence  for  the  existence  of 
each  feature;  in  the  method  of  Jiang  et  al  (9),  the  location  of 
each  microcalcification  was  determined  manually.  This 


396 


Academic  Radiology,  Vol  7,  No  6,  June  2000 


COMPUTER-AIDED  DIAGNOSIS  OF  MAMMOGRAPHIC  REGIONS 


amount  of  human  involvement  likely  makes  these  techniques 
impractical  for  clinical  use. 

Chan  et  al  (20)  restricted  human  involvement  to  identi¬ 
fying  in  each  view  an  ROI  enclosing  a  mass.  This  level  of 
human  involvement  is  likely  to  be  practical  during  clinical 
applications,  because  finding  ROIs  is  part  of  the  usual  pro¬ 
cedure  for  interpreting  mammograms.  As  in  the  method  of 
Getty  et  al  (19),  the  radiologist-reader  in  the  method  of  Chan 
et  al  (20)  was  asked  to  make  a  final  estimate  regarding  the 
likelihood  of  malignancy  in  a  presented  mammogram  after 
first  performing  an  unaided  estimation  and  then  receiving 
the  computer’s  estimate.  In  that  study,  the  performance  of 
the  computer  alone  was  comparable  to  that  of  the  best  per¬ 
forming  unaided  radiologist-reader  and  to  the  performance 
of  six  aided  radiologist-readers  as  a  group. 


MATERIALS  AND  METHODS 


A  retrospective  study  of  the  mapped-database  system  was 
performed  during  June  1998.  The  ROIs  included  in  this 
study  were  restricted  to  those  exhibiting  microcalcifications. 
For  this  study,  we  acquired  a  database  of  200  ROIs  from  138 
proved  cases.  Biopsy  results  were  included  with  each  ROI. 
Four  radiologists  from  the  clinical  faculty  of  the  Department 
of  Radiology  at  King/Drew  Medical  Center  (KDMC)  read 
the  digitized  images  in  this  study.  None  of  these  radiologists 
was  familiar  with  the  cases,  and  all  of  them  were  certified  for 
mammography  by  the  U.S.  Food  and  Drug  Administration. 
Their  postresidency  experience  in  reading  mammograms 
was  29  years,  10  years,  5  years,  and  6  months;  these  readers 
are  referred  to  as  Rj,  R2,  R3,  and  R^,  respectively. 

The  ROIs  in  the  database  were  excised  from  digitizations 
of  film  mammograms  provided  by  KDMC  and  by  the  Univer¬ 
sity  of  California  at  Los  Angeles  (UCLA)  Iris  Cantor  Center 
for  Breast  Imaging.  In  all  cases  associated  with  these  mammo¬ 
grams,  either  biopsy  or  follow-up  with  subsequent  mammog¬ 
raphy  was  performed.  From  these  mammograms,  all  138  cases 
that  revealed  microcalcifications  were  selected  for  this  study. 
Fifty-two  cases  were  from  KDMC,  and  86  were  from  UCLA. 
The  selected  mammograms  were  digitized  at  KDMC  on  a 
scanner-digitizer  (Lumiscan  85;  Lumisys,  Sunnyvale,  Calif) 
at  a  pixel  width  of  50  pm  and  a  pixel  depth  of  12  pm. 

For  each  film  mammogram,  the  location  of  an  abnor¬ 
mality  containing  a  cluster  of  microcalcifications  was  pro¬ 
vided  in  the  medical  record  that  accompanied  the  image. 
From  this  information,  we  constructed  a  512  x  512-pixel 
ROI  enclosing  each  designated  cluster  of  microcalcifica- 
tions.  This  construction  yielded  160  ROIs,  No  two  ROIs  in 
this  set  were  views  of  the  same  lesion.  To  each  ROI  we  ap¬ 


plied  an  automatic  microcalcification  detector  and  segmenter 
that  was  developed  during  an  earlier  study  (7).  An  additional 
40  normal  ROIs  were  also  added  to  this  database,  thereby 
making  a  total  of  200  ROIs  in  this  study.  These  normal  ROIs 
were  included  to  ensure  that  readers  would  have  the  impres¬ 
sion  that  not  all  ROIs  in  the  set  used  to  test  the  system  (de¬ 
scribed  later)  were  abnormal.  The  normal  ROIs  were  ob¬ 
tained  from  cases  involving  patients  who  did  not  undergo 
biopsy  and  were  not  recalled  during  a  period  of  at  least  1 8 
months.  Within  each  of  these  cases,  the  normal  ROI  was 
selected  arbitrarily. 

Of  the  abnormal  ROIs  (ie,  those  containing  microcalcifi¬ 
cations),  64  were  proved  at  biopsy  to  be  benign,  49  were 
proved  at  biopsy  to  be  malignant,  and  47  were  proved  at  fol¬ 
low-up  to  be  benign  (ie,  both  the  ROI  and  the  follow-up 
findings  did  not  produce  a  recommendation  for  biopsy).  At 
least  18  months  elapsed  between  two  successive  examina¬ 
tions,  and  none  of  the  radiologists  on  our  reading  panel  had 
seen  these  ROIs  before  this  study.  One  radiologist  (R^)  parti¬ 
tioned  the  200  ROIs  into  groups  such  that  ROIs  in  the  same 
group  were  visually  similar.  We  refer  to  these  groups  as  per¬ 
ceptual  groups.  No  restriction  was  placed  on  the  number  of 
perceptual  groups,  although  the  radiologist  (Rj)  was  in¬ 
formed  that  eight  groups  were  formed  by  another  radiologist 
during  another  set  of  ROIs  in  an  earlier  experiment  (21).  The 
radiologist  (Rj)  partitioned  the  200  ROIs  into  the  following 
12  groups:  (a)  amorphous  with  mass,  (b)  lobular  and  ductal, 
(c)  amorphous,  (d)  pleomorphic  and  scattered,  (e)  lobular, 
(/)  granular,  (g)  casting,  (h)  punctate  scattered,  (0  linear  duc¬ 
tal,  (/)  vesicular,  (k)  oil  cyst,  and  (0  no  visible  abnormality. 

Computer-aided  reading  of  the  test  set  by  the  reading 
panel  was  performed  more  than  a  month  after  partitioning  of 
the  ROIs  into  perceptual  groups.  The  intent  of  this  1 -month 
delay  was  to  suppress  the  memory  of  the  200  ROIs  in  the  ra¬ 
diologist  who  partitioned  them  into  the  perceptual  groups. 

The  database  was  then  divided  into  two  sets,  D  and  T,  each 
of  which  consisted  of  100  ROIs.  D  was  used  for  the  design  of 
the  diagnostic  system,  and  T  was  used  for  testing  of  the  sys¬ 
tem.  The  160  abnormal  ROIs  were  randomly  partitioned 
several  times  into  two  equal  parts,  and  T^,  until  a  (D^,  T^) 
pair  was  found  such  that  and  each  contained  at  least  two 
ROIs  from  each  perceptual  group.  (Ensuring  representation 
of  at  least  two  ROIs  from  each  perceptual  group,  however, 
may  have  biased  the  diagnostic  system  somewhat  in  favor  of 
the  neural  network.  On  the  other  hand,  not  achieving  this  rep¬ 
resentation  would  have  biased  the  study  against  the  neural 
network.)  consisted  of  54  benign  and  26  malignant  ROIs. 

consisted  of  57  benign  and  23  malignant  ROIs.  Because  no 
two  ROIs  were  images  of  the  same  lesion,  all  lesions  in  T 
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Table  1  ^ ^  ■ 

Features  Extracted  from  Microcalcifications  and  Regions  of  Interest 


Feature 


Area 

Perimeter 


Symbol  _ Formula _ 

A  Number  of  pixels  in  the  calcification  labeled  >  0 

p  Number  of  pixels  in  central  boundary  (label  =  0) 


Irregularity 


A/P2 


Mean  intensity 


Mi  Average  brightness  of  pixels  labeled  >  0 


Standard  deviation  Si 

of  intensity 

Mean  of  exterior  ME 

intensity 

Standard  deviation  of  SE 
exterior  Intensity 

Contrast  C 


Standard  deviation  of  pixels  labeled  >  0 

Average  brightness  of  pixels  within  5  pixels  from 
the  calcification 

Standard  deviation  of  pixels  within  5  pixels  from 
the  calcification 

\Mi  -  S/|  -  Mi 


Sharpness  of  SH 

boundary 

Inner  shell  contrast  B, 
Center  shell  contrast 
Outer  shell  contrast 
Exterior  shell  contrast 
Concavity  index  Ci 

Shape  signature  SG 

Aspect  ratio  B 

Count  N 

Structural  index  S 


Average  change  in  brightness  between  pixels 
labeled  0  or  1  and  those  labeled  -1  or  -2 

Average  change  in  brightness  between  pixels 
labeled  2  and  those  labeled  1 

Average  change  in  brightness  between  pixels 
labeled  1  and  those  labeled  0 

Average  change  in  brightness  between  pixels 
labeled  0  and  those  labeled  -1 

Average  change  in  brightness  between  pixels 
labeled  -1  and  those  labeled  -2 

Area  of  region  between  the  calcification  and  its 
convex  hull 

where  r  is  radial  distance 
of  boundary  from  centroid,  R  is  the  mean  of  r, 
and  0  is  orientation  of  r 

Ratio  of  maximum  central  diameter  to  minimum 
central  diameter 

Number  of  calcifications 

1  -d-sj{N jAC)  ,  where  d is  the  average  dis¬ 
tance  of  the  closest  calcification  and  AC  is 
area  spanned  by  cluster 


Clinical  Importance 

Large  calcification  may  suggest  a  benign 
abnoriiiality. 

Large  calcification  may  suggest  a  benign 
abnormality. 

irregular  boundary  may  suggest  a  malignant 
abnormality. 

Bright  calcification  may  suggest  a  benign 
abnormality. 

Large  Si  may  suggest  an  Irregular  density 
and  malignancy. 

Large  ME  may  suggest  a  dense  and  malig¬ 
nant  tissue. 

Large  SE  may  suggest  a  malignant  tissue. 

Large  contrast  may  suggest  a  benign  abnor¬ 
mality. 

Large  SH  may  suggest  a  benign  abnormality. 

Large  B^  may  suggest  a  benign  abnormality. 

Large  may  suggest  a  benign  abnormality. 

Large  B_^  may  suggest  a  benign  abnormality. 

Large  B,^  may  suggest  a  benign  abnormality. 

Large  Ci  may  suggest  a  malignancy. 


A  large  SG  may  suggest  benign  tissue. 


Large  R  may  suggest  a  malignancy. 

Large  count  may  suggest  malignancy. 

Large  S  indicates  nonuniform  distribution  of 
microcalcifications  and,  hence,  may 
suggest  malignancy. 


were  distinct  from  all  lesions  in  D.  The  40  normal  ROIs  were 
partitioned  several  times  into  two  equal  parts,  and  until 
a  (D^,  Tj)  pair  was  found  such  that  and  T^had  similar  dis¬ 
tributions  in  the  mapped  feature  space.  The  final  design  set  D 
was  the  union  of  and  The  final  test  set  T  was  the  union 


of  and  These  procedures  were  performed  to  ensure  the 
design  set  was  representative  of  the  types  of  abnormalities  oc¬ 
curring  in  the  test  set.  As  mentioned,  the  normal  ROIs  were 
included  to  ensure  the  readers  would  have  the  impression  that 
not  all  ROIs  in  T  were  abnormal.  Because  diagnostic  (rather 
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Figure  1,  Numeric  labeling  for  the  pixels  of  a  digital  model  repre¬ 
senting  a  typical  microcalcification.  The  label  of  each  pixel  is  the 
number  of  pixel  displacements  of  that  pixel  from  the  medial  axis  of 
the  boundary  of  the  microcalcification.  Thus,  0  denotes  a  pixel  on 
the  medial  axis  of  the  boundary,  1  a  pixel  just  inside  the  boundary, 
-1  a  pixel  just  outside  the  boundary,  and  -2  a  pixel  just  outside 
the  “1  region. 

than  detection)  efficacy  was  paramount  in  this  study,  the 
mapped-database  system  in  this  study  was  designed  to  ana¬ 
lyze  only  the  abnormal  ROIs.  Consequently,  readings  of  the 
normal  ROIs  were  not  included  in  our  calculations  of  speci¬ 
ficity,  sensitivity,  and  other  measures  of  performance. 

From  each  abnormal  ROI,  18  candidate  features  were  ex¬ 
tracted.  Each  candidate  feature  was  designed  to  reflect  one  or 
more  properties  of  calcifications  (or  groups  of  calcifications) 
that  are  meaningful  to  the  radiologist.  These  features  were 
based  on  the  American  College  of  Radiology’s  Breast  Imag¬ 
ing  Recording  and  Data  System  (BI-RADS)  descriptors  for 
interpreting  mammograms,  the  book  by  Lanyi  (22),  and  dis¬ 
cussions  with  the  radiologists  on  our  panel.  Unfortunately, 
most  BI-RADS  descriptors  are  too  subjective  for  encoding 
into  computer-executable  formulas.  Consequently,  the  fea¬ 
tures  we  extracted  were  inspired  by,  but  not  necessarily  a  di¬ 
rect  implementation  of,  the  BI-RADS  descriptors.  A  formula¬ 
tion  and  brief  clinical  justification  for  each  of  these  features 
are  given  in  Table  1..  In  the  first  two  columns  of  this  table,  a 
name  and  a  mathematical  symbol  denote  each  feature.  A  for¬ 
mula  for  each  feature  is  given  in  the  third  column.  The  fourth 
column  contains  a  brief  description  of  the  clinical  importance 
of  the  feature.  Some  of  the  formulas  refer  to  numerical  pixel 


labels  -2,  -1, 0,  and  I.  The  label  on  each  pixel  P  is  the  num¬ 
ber  of  pixel  displacements  of  P  from  the  boundary  of  a  calci¬ 
fication.  The  label  0  denotes  a  pixel  on  the  boundary,  1  de¬ 
notes  a  pixel  just  inside  the  boundary,  -1  denotes  a  pixel  just 
outside  the  boundary,  and  -2  denotes  a  pixel  just  outside  the 
“1  region.  These  labels  are  illustrated  in  Figure  1,  which 
shows  a  digital  model  of  a  typical  microcalcification. 

Starting  from  this  set  of  candidate  features,  a  genetic  al¬ 
gorithm  searched  for  those  subsets  of  features  that  were  most 
effective  in  discriminating  malignant  from  benign  cases  in 
the  database.  This  algorithm  was  designed  in  accordance 
with  the  genetic  feature  selector  described  by  Siedlecki  and 
Sklansky  (23).  In  this  algorithm,  each  candidate  subset  is 
represented  by  a  binary  string  (ie,  a  sequence  of  Os  and  Is). 
The  value  (0  or  1)  of  the  nth  element  of  the  string  indicates 
whether  the  nth  feature  belongs  to  the  candidate  subset.  We 
refer  to  this  string  as  a  chromosome,  A  population  of  100 
chromosomes  was  transformed  by  an  evolutionary  process 
consisting  of  mutation,  mating,  and  reproduction  to  form  a 
sequence  of  new  populations  or  “generations”  under  the 
guidance  of  a  penalty  function  that  accounted  for  the  /:-near- 
est-neighbor  discriminability  of  the  feature  vectors  in  each 
feature  subset  and  for  the  number  of  features  in  the  subset. 
The  sizes  of  the  new  populations  were  kept  constant  at  100. 
This  evolutionary  process  also  optimized  the  value  of  k.  To 
ensure  an  acceptably  low  rate  of  misdiagnosed  cancers,  the 
penalty  function  gave  seven  times  as  much  weight  to  misdi¬ 
agnosing  a  cancer  as  to  recommending  a  biopsy  that  was 
benign.  (The  weight  was  chosen  empirically  so  that  the 
nearest-neighbor  classifier  would  produce  only  one  or  two 
missed  cancers  on  the  data  set.)  This  algorithm  selected 
the  following  four  features:  {a)  mean  area,  (b)  mean  as¬ 
pect  ratio,  (c)  mean  irregularity  (ie,  noncircularity),  and 
{d)  number  of  microcalcifications.  Another  algorithm  ex¬ 
tracted  these  four  features  from  each  ROI,  thus  forming  a 
four-dimensional vector  as  a  descriptor  of  that  ROI. 

A  five-layer  neural  network  was  constructed  in  accor¬ 
dance  with  the  architecture  and  the  design  principles  de¬ 
scribed  by  Omes  and  Sklansky  (24).  Each  node  in  this  archi¬ 
tecture  represented  an  artificial  neuron  that  consisted  of  a 
weighted  summation  followed  by  a  sigmoidal  activation 
function.  The  first  layer  consisted  of  four  input  neurons,  one 
for  each  selected  feature.  The  second  and  third  layers  con¬ 
sisted  of  three  and  two  neurons,  respectively,  to  reduce  the 
dimensionality  of  feature  space.  The  two-neuron  layer  pro¬ 
duced  x  and  y  coordinates  (one  from  each  of  the  two  neu¬ 
rons)  to  represent  each  ROI  as  a  dot  on  a  relational  map.  For 
each  ROI  in  the  design  set,  this  dot  was  labeled  to  indicate 
whether  the  biopsy  was  benign  or  malignant.  The  two 
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neurons  in  the  fifth  layer  classified  each  feature  vector  as 
either  “biopsy  recommended”  or  “biopsy  not  recom¬ 
mended  ”  (The  neuron  producing  the  larger  of  the  two  out¬ 
puts  determined  the  class  of  the  feature  vector.)  By  entering 
every  permissible  pair  of  map  coordinates  at  the  input  to  the 
fourth  layer,  the  fifth  layer  thus  produced  a  relational  map 
in  which  all  pixels  were  labeled  as  either  biopsy  recom¬ 
mended  or  biopsy  not  recommended. 

Using  the  four  features  selected  by  the  genetic  algorithm, 
the  neural  network  was  trained  by  backpropagation  (ie,  a 
form  of  gradient  descent  in  the  space  formed  by  the  weights 
at  the  inputs  to  the  neurons)  on  the  design  set  thereby  pro¬ 
ducing  the  trained  neural  network  NN(D^)  (4).  In  accordance 
with  our  design  philosophy  for  this  study,  the  neural  network 
was  trained  only  on  abnormal  ROIs.  The  biopsy  outcomes 
(ie,  benign  and  malignant)  in  the  database  were  the  desired 
output  classes  in  the  trained  neural  network,  which  produced 
a  two-dimensional  map  of  and  a  near-optimal  partition  of 
this  map  into  two  decision  regions.  These  decision  regions 
were  associated  with  the  decisions  of  biopsy  recommended 
and  biopsy  not  recommended.  The  training  was  stopped 
when  the  mean  square  error  converged  to  a  nearly  constant 
value.  This  design  strategy,  and  the  relatively  low  error  rate  of 
the  neural  network  on  the  test  set  T,  encouraged  us  to  believe 
that  the  network  was  then  adequately  trained. 

The  boundary  separating  the  decision  regions  is  referred 
to  as  a  decision  curve.  The  decision  curve  is  substantially 
smoother  than  the  variability  of  the  data,  which  is  the  result 
of  the  averaging  produced  by  the  backpropagation  training 
and  the  small  number  of  neurons  compared  with  the  number 
of  feature  vectors  in  the  training  set.  The  neural  network 
tends  to  produce  decision  regions  that  are  responsive  to  sta¬ 
tistical  models  of  the  training  data  rather  than  to  the  training 
data  itself  Specifically,  the  training  algorithm  minimized  a 
weighted  sum-of-squared-error  function,  which  yielded  out¬ 
puts  that  were  estimates  of  Bayesian  posterior  probabilities 
that  the  input  vector  belonged  to  the  corresponding  class  (ie, 
biopsy  recommended  or  biopsy  not  recommended).  Thus, 
the  decision  curve  produced  by  the  neural  network  tends  to 
“generalize”  the  training  data  and,  thereby,  to  make  good  de¬ 
cisions  on  future  data  not  included  in  the  training  set  (25). 

The  trained  neural  network  displayed  the  query  and  the 
ROIs  in  as  dots,  with  each  dot  being  located  by  the  x  and 
y  coordinates  produced  by  the  two  neurons  (one  coordinate 
from  each  neuron)  at  the  two-neuron  layer  of  the  network. 
We  refer  to  the  space  spanned  by  x  and  y  as  the  mapped  fea¬ 
ture  space  and  to  the  map  of  as  a  mapped  database.  The 
map  of  and  its  decision  regions  are  depicted  in  Figure  2. 
We  call  this  a  relational  map.  Here,  the  black  region  repre- 


Flgure  2.  Relational  map  of  design  set  D  and  the  query  ROI.  Each 
malignant  case  is  labeled  X,  and  each  benign  case  is  labeled  O. 
The  query  ROI  is  labeled  ■. 


sents  biopsy  recommended,  and  the  white  region  represents 
biopsy  not  recommended.  The  decision  curve  is  the  bound¬ 
ary  between  the  black  and  the  white  regions.  The  symbols  O 
and  X  denote  benign  and  malignant  feature  vectors,  respec¬ 
tively.  The  symbol  ■  denotes  a  query. 

A  weakness  of  the  mapped-database  system  is  that  it  does 
not  reveal  the  quantitative  relationships  between  the  mapped 
feature  space  and  the  unmapped  features.  This  weakness  is 
compensated  for,  however,  by  the  ability  of  the  relational 
map  to  cluster  visually  and  diagnostically  similar  ROIs  and 
by  the  map’s  representation  of  a  multidimensional  decision 
surface  as  a  decision  curve  (or  as  several  disjoint  decision 
curves)  in  the  mapped  feature  space.  The  clustering  helps  the 
radiologist  to  find  ROIs  that  are  visually  similar  to  the  query, 
and  the  decision  curve  helps  the  radiologist  to  determine  the 
confidence  of  the  neural  network  in  its  recommendations  ei¬ 
ther  for  or  against  performing  biopsy.  Further  insight  into 
the  relationships  between  the  mapped  feature  space  and  the 
original  features  can  be  obtained  by  making  available  the  nu¬ 
merical  values  of  the  original  features  of  each  mapped  ROI 
to  the  radiologist-user. 

The  user  interface  presented  on  the  computer  monitor  is  il¬ 
lustrated  in  Figure  3.  This  interface  guides  the  radiologist-user 
through  the  steps  of  diagnosis.  Identification  numbers  in  this 
figure  (eg,  011318)  were  constructed  so  that  patients  could  not 
be  identified  from  the  information  in  the  figure.  The  mammo- 
graphic  image  in  the  upper  left  is  an  enlargement  of  the  ROI 
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Roianoftai  Map 


1  W04<2n» 

ID: 091 04 
Biopsy: 
Benign 

2  Q629<(21M2) 

ID:  09104 
Biopsy: 
Benign 

3  223t*(321M8) 


Figure  3.  User  interface  for  mammographic  diagnosis. 


of  a  query  mammogram.  Digital  magnification  and  negative- 
to-positive  inversion  were  provided  as  options  on  the  user  in¬ 
terface.  In  response  to  a  prompt  from  the  interface,  the  radi¬ 
ologist  first  determines  an  “unaided”  diagnosis  of  this  ROI  and 
reports  it  in  the  BI-RADS  code  (26)  by  clicking  N,  B,  P,  S,  or 
M,  which  denote  normal,  benign,  probably  benign,  suspicious 
(possibly  malignant),  and  almost  definitely  malignant,  respec¬ 
tively.  In  this  study,  we  assumed  that  N,  B,  and  P  corresponded 
to  biopsy  not  recommended  and  that  S  and  M  corresponded  to 
biopsy  recommended.  This  assumption  conformed  to  the  clini¬ 
cal  practice  at  KDMC.  We  instmcted  each  reader  to  produce 
the  unaided  diagnosis  as  if  it  were  the  final  diagnosis  for  the 
examination.  No  time  limit  was  imposed  on  this  diagnosis. 

In  this  study,  the  diagnostic  process  began  by  establish¬ 
ing  an  unaided  diagnosis,  which  was  followed  immediately 
by  establishing  an  aided  diagnosis  (analogous  to  double 
reading).  We  assumed  that  the  operating  point  (ie,  the  sub¬ 
jective  decision  threshold  for  a  BI-RADS  symbol)  of  the 
unaided  radiologist  might  be  affected  by  that  radiologist’s 
anticipation  of  an  aided  diagnosis,  but  that  the  receiver  op¬ 
erating  characteristic  (ROC)  of  the  unaided  diagnosis  was 
unaffected  by  that  anticipation. 

Immediately  after  the  unaided  diagnosis  was  established, 
the  relational  map  was  presented  to  the  radiologist.  This  map 
contained  the  mapped  database,  mapped  query,  and  distinc¬ 
tively  colored  decision  regions.  Thumbnail  images  of  six 


ROIs  near  the  query  in  the  mapped  feature  space  were  pre¬ 
sented  at  the  right  of  the  monitor  screen.  From  these  images, 
the  radiologist  retrieved  for  review  those  with  a  visual  appear¬ 
ance  that  seemed  to  be  most  similar  to  that  of  the  query.  This 
review  was  facilitated  by  an  enlarged  view  of  the  retrieved 
ROI  and  the  accompanying  biopsy  reports.  On  the  basis  of 
this  interaction,  the  radiologist  selected  a  BI-RADS  score  for 
the  ROI.  Again,  the  reader  was  not  restricted  by  a  time  limit 
for  establishing  this  diagnosis,  and  both  magnification  and  in¬ 
version  were  provided  as  options  on  the  user  interface. 

To  evaluate  the  diagnostic  improvement  provided  by  the 
mapped-database  diagnostic  system,  we  computed  the  ROC 
of  each  unaided  radiologist,  of  each  computer-aided  radiolo¬ 
gist,  and  of  the  computer  alone.  Each  ROC  accounted  for  the 
BI-RADS  responses  of  the  radiologists  and  for  the  outcomes 
of  the  biopsies.  The  diagnostic  performance  of  each  radiolo¬ 
gist  was  measured  by  the  area  under  the  ROC  curve,  which 
is  an  estimate  of  the  probability  for  a  correct  decision  in  a 
forced  choice  between  two  ROIs,  one  of  which  is  malignant 
and  the  other  of  which  is  benign  (27).  Thus,  must  lie  be¬ 
tween  a  minimum  of  0.5  (corresponding  to  an  unbiased  ran¬ 
dom  guess)  and  a  maximum  of  1  (perfect  performance). 

We  also  computed  the  average  performance  of  the  four  ra¬ 
diologists  when  unaided  by  the  mapped-database  system  and 
when  aided  by  the  mapped-database  system.  LABMRMC 
(C.E.  Metz,  LABMRMC  l.OB,  beta  version  3,  University  of 
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Figure  4.  ROC  curves  for  radiologist  R,  and  for  the  computer 
alone. 

Chicago,  Chicago,  Ill)  (28)  was  employed  to  calculate  the 
maximum  likelihood  estimates  for  the  parameters  of  binor¬ 
mal  models  and  the  statistical  significance  of  the  change  in 
areas  under  the  ROC  curves  for  the  unaided  and  the  aided 
reading  modalities.  The  ROC  curves  representing  the  aver¬ 
age  performance  of  the  unaided  radiologists  were  computed 
by  averaging  their  estimated  slopes  and  intercepts  in  normal 
deviate  space.  This  same  process  was  also  performed  for  the 
aided  radiologists. 

The  mapped-database  diagnostic  system  was  designed 
and  implemented  at  the  University  of  California,  Irvine,  on 
an  IBM-PC-compatible  computer  with  a  66-MHz  Intel  486 
processor,  32  MB  of  random  access  memory,  and  a  17-inch 
CTX  monitor  with  1,280  x  1,024  pixels  running  on  a  Micro¬ 
soft  (Redmond,  Wash)  Windows  95  operating  system.  The 
software  development  environment  was  Microsoft  Visual 
C++  4.0  for  image-processing  functions  and  Microsoft  Vi¬ 
sual  Basic  5.0  for  the  user  interface.  Testing  of  this  system 
was  performed  at  KDMC  on  another  IBM-PC— compatible 
computer  with  a  233-MHz  Pentium  II  processor,  64  MB  of 
random  access  memory,  and  a  17-inch  monitor  (Vivitron; 
Gateway  2000,  Sioux  City,  SD)  with  1,280  x  1,024  pixels 
running  on  a  Microsoft  Windows  95  operating  system. 


RESULTS 


The  ROC  curves,  labeled  by  their  values,  are  shown 
in  Figures  4-8.  The  statistical  significance  of  the  changes 


False- Positive  Fraction 

Figure  5.  ROC  curves  for  radiologist  Rj  and  for  the  computer 
alone. 


Figure  6.  ROC  curves  for  radiologist  R3  and  for  the  computer 
alone. 


in  these  values  is  reported  in  Table  2.  The  columns  of 
values  for  the  unaided  radiologist,  the  aided  radiologist, 
and  the  computer  alone  are  labeled  A,  B,  and  C,  respec¬ 
tively.  The  column  labeled  A-B  denotes  the  values  of  P  as¬ 
sociated  with  the  increase  in  A^  obtained  by  replacing  an 
unaided  radiologist  with  the  corresponding  aided  radiolo¬ 
gist.  The  values  of  P  listed  in  Table  2  are  two-tailed  and 
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Figure  7.  ROC  curves  for  radiologist  and  for  the  computer 
alone. 


Table  2 

Comparison  of  ROC  Curves  for  the  Unaided  Radiologist,  the 
Aided  Radiologist,  and  the  Computer  Alone  _ . 


Reader 

P 

A,  Unaided 

B,  Aided 

C,  Computer 
Alone 

A-B, Two- 
tailed 

0.7150 

0.8354 

0.7533 

0.0157 

0.7179 

0.8814 

0.7533 

0.0013 

R3 

0.6277 

0.7538 

0.7533 

0.0015 

R4 

0.7157 

0.8077 

0.7533 

0.0636 

were  calculated  under  the  assumption  that  the  distribution  of 
each  estimated  change  in  A^,  divided  by  the  estimated  stan¬ 
dard  error  of  that  change,  arose  from  a  normal  distribution. 
These  values  of  P  were  computed  by  using  the  ROCKIT 
software  (C.  E.  Metz,  ROCKFIT  0.9B,  beta  version,  Univer¬ 
sity  of  Chicago,  Chicago,  III,  1998). 

The  information  in  Figures  A-1  indicates  that  (a)  every 
aided  radiologist  outperformed  the  same  unaided  radiologist, 
{b)  no  unaided  radiologist  outperformed  the  computer  alone, 
and  (c)  three  aided  radiologists  outperformed  the  computer 
alone  and  the  fourth  aided  radiologist  (R3)  performed  ap¬ 
proximately  equal  to  the  computer  alone.  The  information 
in  Table  2  indicates  that  the  A^  values  of  the  unaided  radi¬ 
ologists  ranged  from  0.6277  to  0.7179,  and  that  those  of 
the  aided  radiologists  ranged  from  0.7538  to  0.8814.  The 
size  of  the  changes  in  A^  among  the  radiologists  ranged  from 


Figure  8.  ROC  curves  for  the  average  performance  of  the  four 
radiologists. 


0.0920  to  0.1635.  Table  2  also  indicates  that  the  A-B  changes 
in  A  (ie,  unaided-to-aided  changes  in  A^  were  statistically 
significant  at  P  <  .002  for  radiologists  R^  and  R3,  at  P  <  .02 
for  radiologist  R^,  and  at  P  <  .07  for  radiologist  R^.  After  ap¬ 
plication  of  the  Bonferroni  correction  for  multiple  compari¬ 
sons,  these  three  P  bounds  became  .008,  .08,  and  .28,  respec¬ 
tively  (29).  Other  changes  in  A^ — ^namely,  A-C  (ie,  unaided- 
to-computer  alone  changes)  and  B-C  (ie,  aided-to-computer 
alone  changes) — were  not  statistically  significant. 

Figure  8  shows  the  average  performance  of  the  four  radi¬ 
ologists  when  unaided  by  the  mapped-database  system  and 
when  aided  by  the  mapped-database  system.  Results  of  the 
LABMRMC  analysis  indicate  that  performance  of  the  aided 
radiologists,  as  a  group,  was  statistically  superior  to  that  of 
the  unaided  radiologists  as  a  group  (P  <  .05). 

At  KDMC,  the  diagnostic  scores  S  and  M  are  convention¬ 
ally  interpreted  as  being  biopsy  recommended,  and  N,  B,  and 
P  are  interpreted  as  being  biopsy  not  recommended.  In  ac¬ 
cordance  with  this  convention,  we  calculated  the  numbers  of 
negative  biopsies  and  misdiagnosed  cancers,  as  well  as  the 
sensitivities  and  specificities  of  each  radiologist,  both  aided 
and  unaided  by  the  mapped-database  system.  These  results 
are  shown  in  Tables  3-5.  Tables  3  and  4  show  the  numbers 
of  recommended  benign  biopsies  and  misdiagnosed  cancers. 
Table  5  shows  the  sensitivities  and  specificities.  The  mapped- 
database  diagnostic  system  helped  every  radiologist  reduce 
the  number  of  benign  biopsies  by  40%  or  more  while  simul¬ 
taneously  reducing  the  number  of  misdiagnosed  cancers.  All 


403 


SKLANSKY  ET  AL 


Academic  Radiology,  Vol  7,  No  6,  June  2000 


Table  3 

Numbers  of  Recommended  Benign  Biopsies  for  the  Unaided 
Radiologist  and  the  Aided  Radiologist _ 


Reader 

Unaided  Radiologist 

Aided  Radiologist 

R, 

18 

11 

R, 

15 

8 

R3 

28 

18 

R. 

18 

10 

NN 

24 

Note.— Number  of  ROIs  from  benign  cases:  57.  NN  =  neural 
network. 


the  radiologists  also  increased  both  their  sensitivities  and 
specificities  through  their  interaction  with  the  mapped-data- 
base  system.  The  statistical  significances  of  these  increases 
ranged  from  P  <  .01  to  P  <  .12  on  the  basis  of  a  comparison 
of  proportions  (30). 

As  mentioned,  the  sensitivities,  specificities,  and  ROCs 
were  calculated  on  the  basis  of  the  160  abnormal  ROIs  and 
the  responses  of  the  panel  of  radiologists  to  them.  The  re¬ 
sponses  of  the  panel  of  radiologists  to  the  20  normal  ROIs 
(the  remaining  20  normal  ROIs  were  in  the  set  Dj^)  were 
excluded  from  these  calculations,  because  in  this  study,  the 
mapped-database  diagnostic  system  was  designed  only  to 
interpret  abnormal  ROIs. 


DISCUSSION 


Figures  4-8  and  Tables  2-5  indicate  that  the  diagnostic 
performances  of  the  aided  radiologists  were  statistically 
significantly  improved  compared  with  those  of  the  unaided 
radiologists.  Table  2  shows  that  the  sizes  of  these  improve¬ 
ments  in  for  the  current  study  are  comparable  to  those  in 
A  found  during  a  recent  study  concerning  computer-aided 
characterization  of  mammographic  masses  (20). 

We  are  also  encouraged  by  the  improved  performance  of 
the  aided  radiologists  over  that  of  the  computer  alone,  as  in¬ 
dicated  in  Figures  4,  5,  7,  and  8  and  in  Table  2,  although  the 
number  of  cases  was  insufficient  to  prove  statistical  signifi¬ 
cance  of  these  improvements.  Table  5  shows  that  the  mapped- 
database  diagnostic  system  enabled  substantial  increases  in 
specificities  and  sensitivities  for  all  the  radiologists  in  our 
panel.  We  conclude  that  for  ROIs  containing  microcalcifica¬ 
tions,  the  mapped-database  diagnostic  system  shows  promise 
in  helping  most  radiologists  to  raise  their  diagnostic  perfor¬ 
mances  substantially  over  their  unaided  performances  while 
providing  relevant  images  from  a  proved  database  to  support 
the  radiologists’  aided  diagnoses.  In  particular,  these  radiolo¬ 


Table  4 


Numbers  of  Misdiagnosed  Cancers  for  the  Unaided 
Radiologist  and  the  Aided  Radiologist 

Reader 

Unaided  Radiologist 

Aided  Radiologist 

R, 

11 

8 

R2 

12 

7 

R3 

7 

4 

R. 

8 

3 

NN 

4 

Note.— Number  of  ROIs  from  malignant  cases:  23.  NN  =  neural 
network. 


gists  may  achieve  substantial  reductions  in  the  number  of 
benign  biopsies  and  misdiagnosed  cancers. 

The  levels  of  postresidency  mammographic  experience 
among  the  radiologists  in  this  study  ranged  from  29  years 
to  6  months  (R^  >  R2  >  R3  >  R4).  Figures  4-7  suggest  that 
all  radiologists  are  likely  to  benefit  substantially  from  ac¬ 
cess  to  the  mapped-database  diagnostic  system  regardless 
of  their  experience.  This  is  important,  because  radiologists 
vary  widely  in  their  diagnostic  skills — even  among  those 
who  rate  themselves  as  “experts”  (31).  Whether  the  size  of 
the  benefit  correlates  with  the  amount  of  experience,  how¬ 
ever,  is  not  clear  from  Figures  4-7.  (It  is  interesting  that  the 
performance  of  the  aided  R2  exceeded  that  of  the  aided  Rj, 
even  though  the  experience  of  R,  was  greater.) 

The  results  of  this  study  suggest  that  a  properly  designed 
interface  between  a  human  reader  and  a  proved  database  of 
mammographic  images  may  enable  less  experienced  readers 
to  exceed  the  performances  of  unaided,  highly  skilled  read¬ 
ers — without  the  need  for  time-consuming  training  on  fea¬ 
ture  extraction  as  exemplified  by  Getty  et  al  (19).  Further¬ 
more,  our  diagnostic  system  represents  an  improvement  over 
earlier  systems  (17,19)  by  automating  the  selection  of  visual 
features. 

This  study  was  limited,  however,  by  the  quality  of  the 
monitors  used  for  viewing  the  images,  the  size  and  compre¬ 
hensiveness  of  the  mammographic  database,  the  lack  of 
full-field  and  earlier  views,  the  restriction  of  the  lesions  to 
microcalcifications,  and  a  possible  “reading-order  effect” 
associated  with  the  close  succession  of  unaided  and  aided 
diagnoses  (32).  Our  next  investigation  of  the  mapped-data¬ 
base  diagnostic  system  will  include  several  enhancements 
to  overcome  these  limitations.  The  images  will  be  viewed 
on  state-of-the-art,  high-resolution  radiographic  monitors, 
and  the  number  of  cases  will  be  increased.  The  tested  im¬ 
ages  will  also  include  full-field  craniocaudal  and  mediolat- 
eral  oblique  or  lateral  views  of  each  breast,  and  the  lesions 
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Table  5 

Statistical  Significance  of  Changes  in  Performance 

Unaided  Radiologist  (%)  Aided  Radiologist  (%)  Change  in  Sensitivity  Change  in  Specificity 


Reader  Sensitivity  Specificity  Sensitivity  Specificity  Change  (%) _ P _ Change  (%) 


R, 

47.8 

73.7 

69.6 

86.0 

-1-21.8 

<.01 

-F12.3 

.12 

1 

65.2 

68.4 

87.0 

82.5 

+2^.Q 

<.01 

-H14.1 

<.05 

2 

R, 

52.2 

68.4 

65.2 

80.7 

-F13.0 

.09 

+12.3 

.07 

3 

R4 

69.6 

50.9 

82.6 

68.4 

-H13.0 

<.05 

+17.5 

<.05 

will  include  both  masses  and  microcalcifications.  In  addi¬ 
tion,  the  aided  and  unaided  diagnoses  will  be  separated  by 
at  least  1  month. 
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Co-chairman  of  Program  Committee  for  SPIE  Conference  on  Hybrid  Image  and 
Signal  Processing,  Orlando,  Florida,  April  1988. 


Reviewer  for  the  following  journals: 

IEEE  Transactions  on  Medical  Imaging 

IEEE  Transactions  on  Pattern  Analysis  and  Machine  Intelligence 

Machine  Vision  and  Applications 

Pattern  Recognition 

Computer  Vision,  Graphics  and  Image  Processing 
International  Journal  of  Pattern  Recognition  and  Artificial  Intelligence 
DBEE  Transactions  on  Systems  Man  and  Cybernetics 
Journal  of  the  Optical  Society  of  America  A 

Reviewer  for  the  following  funding  agencies: 

National  Institutes  of  Health 

National  Science  Foundation 

U.S.  Army  Medical  Research  and  Materiel  Command 

University  of  California  MICRO  program 

National  Research  Council  of  Canada 


Reviewer  of  books  for  the  following  publishers: 

lOP  Publishing  Ltd.,  Philadelphia 
Springer- Verlag,  New  York 
John  Wiley,  New  York 
Addison-Wesley,  Reading,  Massachusetts 


Ph.  D.  Advisees  (names,  title  of  dissertation,  year  of  graduation,  current  position  if  known) : 

Brian  Lofy,  “Segmenting  and  Matching  of  Aerial  Images  Acquired  by  Different 
Sensors,”  1999:  Senior  Engineer,  Marconi  Corporation,  San  Diego,  California. 

Eric  Y.  Tao,  “Database-Aided  Diagnosis  in  Digital  Mammography,”  1998:  Assistant 
Professor,  Institute  of  Communication  Science  and  Technology,  California  State 
University,  Monterey  Bay,  California 

Mark  Vriesenga,  “Genetic  design  of  pattern  classifiers,”  1995:  Senior  Engineer, 
Marconi  Corporation,  San  Diego,  California. 

L. V.  Tran,  "Quantitative  Biplane  Angiography",  1991:  Research  Engineer,  Northrop 
Corporation,  Anaheim,  California 

Y.  Moon,  'Multicomputer  architectures  for  image  analysis,"  1990:  Assistant 
Professor,  Korea  Academy  of  Industrial  Technology,  Seoul,  Korea 

D.  Gutfinger,  "Mixed  Adaptation  ~  a  Theory  for  Designing  Robust  Classifiers,"  1990: 
Student  at  UCI  College  of  Medicine 

Y.  Park  ,  "Linear  Tree  Classifiers"  1990,  Assistant  Professor,  Kyung  Hee  University, 
Suwon  Campus,Yongin,  Kyungkido,  Korea 

W.  W.  Siedlecki,  "Feature  selection  for  large  scale  classifiers,"  1988:  Vice  President 
for  Research,  P-Logic  Corporation 

K.  Kitamura,  "Estimating  the  transverse  areas  and  boundaries  of  coronary  arteries  in 
three  dimensions,"  1987:  Research  Engineer,  Nippon  Steel  Corporation 

P.  E.  Chandler,  "A  multiple  scattering  model  for  pulse  propagation  within  velocity 
inhomogeneous,  lossy,  and  refractive  media,"  1986 

T.  V,  Nguyen,  "Three-dimensional  reconstruction  of  the  medial  axes  of  coronary 
arteries  from  single-view  angiograms,"  1986:  President,  Multisignal  Technology 
Corporation;  Professor,  California  State  University,  Long  Beach,  California 

1.  Foroutan,  "Feature  selection  for  piecewise  linear  classifiers,"  1985:  Manager  of 
Circuit  Automation,  Hughes  Aircraft  Co. 

M.  Hashimoto,  "Estimation  of  multiple-order  derivatives  for  edge  detection,"  1984: 
member  of  Japan  Defense  Force 


D.  Petkovic,  "A  computer  system  for  coarse-fine  detection  of  pulmonary  nodules  in 
chest  radiographs,"  1983:  Manager  of  Department  of  Advanced  Algorithms, 
Architectures,  and  Applications,  IBM  Almaden  Research  Center 

L.  A.  Ferrari,  "Recursive  binary-valued  image  filters,"  1980:  Associate  Professor  of 
Electrical  and  Computer  Engineering,  Chairman  of  Department  of  Electrical  and 
Computer  Engineering,  UC  Irvine 

E.  J.  Pisa,  "Computing  the  geometry  of  the  rib  cage  from  two  chest  radiographs," 
1979:  Vice  President  of  Research  and  Development,  Ivac  Corporation 

H.  Wechsler,  "Automatic  Detection  of  Rib  Contours  in  Chest  Radiographs,"  1975: 
Professor  of  Computer  Science,  George  Mason  University 

D.  H.  Ballard,  "Hierarchic  Recognition  of  Tumors  in  Chest  Radiographs,"  1974: 
Professor  of  Computer  Science,  University  of  Rochester 

P.  J.  Nahin,  "A  parallel  machine  for  describing  and  classifying  silhouettes,"  1972: 
Associate  Professor  of  Electrical  Engineering,  University  of  New  Hampshire 

P.  M.  Merryman,  "Dynamics  of  multidimensional  Markov  learning,"  1972:  Senior 
System  Engineer,  Ultrasystems,  Inc. 

G.  N.  Wassel,  "Training  a  linear  classifier  to  optimize  the  error  probability,"  1972: 
Professor  of  Electrical  Engineering,  Cal  Poly  University,  San  Luis  Obispo,  California 

H.  R.  Ramanujam,  "A  stopping  rule  for  threshold  learning,"  1970:  Principal  Engineer, 
McDonnell  Douglas  Corp. 


M.  S.  Advisees; 

Chester  Omes,  “  A  Neural  Classifier  for  Database-Aided  Decisions,”  1997. 

J.  Ribas-Corbera,  "Image  Data  Compress  and  Interframe  Interpolation  of  Cinematic 
Sequences,"  1992. 

M.  R.  Vriesenga,  "Colored  Illumination  for  Maximizing  Contrast  in  Machine 
Vision,"  1991. 

R.  Agarwal,  "Optical  Flow  from  Clusters  in  Velocity  Space,"  1991. 

E.  M.  Hertzberg,  "Quantitative  Analysis  of  Magnetic  Resonance  Images,"  1991. 


J.  B.  Bartels,  "Analysis  of  Random  and  Systematic  Errors  in  T,  Measurement  with 
Gradient  Echo  Magnetic  Resonance  Imaging,"  1990. 

S.  R.  Reuman,  "Computing  Albedo,  Illumination,  and  Optical  Flow  with 
Spacetime-Adaptive  Bandpass  Filtering,"  1988. 

J.  J.  Duffy,  "Time  Varying  System  Feedback  Design,"  1988. 

N.  niev,  "Failure  Isolation  in  Linear  Stochastic  Systems  --  the  Associative  Recall 
Approach,"  1987. 

R.  F.  DeShetler,  "Finding  the  Polyhedra  Formed  by  the  Hyperplanes  of  a  Piecewise 
Linear  Classifier,"  1986. 

N.  P.  DeMesa,  "Petii  Net  Analysis,"  1986. 

P.  E.  Chandler,  "A  Multiple  Scattering  Model  for  Pulse  Propagation  within  Velocity 
Inhomogeneous,  Lossy,  and  Refractive  Media,"  1986. 

S.  V.  Hung-Leo,  "Computer-Aided  Design  of  Multiple-Class  Piecewise  Linear 
Classifiers,  1983. 

F.  Towfiq,  "Piecewise  Linear  Detection  of  Curved  Edges,  1980. 

M.  D.  McNeil,  "Digital  Preprocessing  of  Low  Contrast,  Wide  Dynamic  Range 
Images,"  1975. 

T.  Miyatake,  "Compute-Film  System  for  Contrast  Enhancement,"  1972. 

D.  V.  Smith,  "Dynamic  Behavior  of  Time-Varying  Threshold  Learning  Processes," 
1968. 

Visiting  Scholars: 

Professor  Minechi  Kudo,  Hokkaido  University,  Sapporo,  Japan,  9  months,  1996. 

Professor  Leon  Bobrowski,  Polish  Academy  of  Science,  Fulbright  Scholar,  Poland,  1 
year,  1994-1995. 

Oliver  Paetz,  RW  Technische  Hochschule,  Germany,  10  months,  1994-1995. 

Matthias  Kroemer,  RW  Technische  Hochschule,  6  months,  1995-1996. 

Ariela  Kamin,  Technion  --  Israel  Institute  of  Technology,  Israel  1990  -1991 


Akio  Shio,  Nippon  Telegraph  and  Telephone  Corporation,  Japan  -  1.5  years  1989- 
1991 

Professor  Kalman  Peleg,  Technion  -  Israel  Institute  of  Technology,  Israel  -  5  months, 
1990 

Professor  Peter  Jensch,  The  Uitiversity  of  Oldenburg,  West  Germany  -6  weeks,  1989- 
1990 

Ryuji  Nishimura,  Hitachi  Ltd.,  Japan  - 1.5  years  1989  - 1990 

Dr.  Thomas  ToIxdorff,  Rheinisch-Westfalischen  Technischen  Hochscule,  West 
Germany  -  4  weeks,  1989 

Dr.  Leon  Bobrowski,  Institute  of  Biocybemetics  and  Biomedical  Engineering, 
Poland,  3  weeks,  1989 

Djordije  Jankovic,  Boris  Kidric  Institute,  Yugoslavia  -  3  months,  1989 
Professor  Mirek  Pawlak,  The  University  of  Manitoba,  Canada  -  2  months,  1989 
Hideaki  Doi,  Hitachi  Ltd,  Japan  - 1  year,  1988-1989 
Professor  Adam  Krzyzak,  Concordia  University,  Canada  - 1  month,  1988. 


EXTRAMURAL  FUNDING  SINCE  1987 


PROGRAM  TITLE 

DATES 

AGENCY  MONETARY  LEVEL 

Database- Aided  Diagnosis 
in  Digital  Mammography 

1999-03 

U.  S.  Army  Medical 
Research  and  Materiel 
Command 

$1,928,000 

Database-Aided 

Telemammography 

2000-02 

California  Telehealth  & 
Telemedicine  Center 

$100,000 

Computer-Aided  Diagnosis 
In  Digital  Mammography 

1997-99 

California  Breast  Cancer 
Research  Program 

$214,000 

Biologically  Inspired 
Intelligent  Classifiers 

1992-96 

NSF 

$  330,000 

Landmark  Detection 


1997-98 

Line  Detection  and  Genetic  1995-96 


Feature  Selection 

Genetic  Algorithms  for  1993-94 

Target  Recognition 

Digital  Video  for  1993-94 

Intelligent  Systems 

Target  Recognition  1992-93 

Computer  Vision  1989-92 

Automated  Tissue  1989-92 

Production 

Neural  Learning  1989-91 

Image  Processing  1989-90 

Architectures 

Computer  Vision  1988-90 

Automatic  Pattern  1987-90 

Classification 

Discretionary 

Research 


Naval  Air  Warfare  Center  $25 ,000 
ONR  $  135,000 

Naval  Air  Warfare  $  20,000 

Center 

NSF  $  25,000 

UCMICRO/Loral  $  53,000 


NTT  $  120,000 

(Nippon  Telegraph  and 
Telephone  Corporation) 

BARD  (Binational  $  129,000 

Agricultural  Research 
&  Development  Fund) 


DARPA 

$ 

300,000 

Rockwell/Micro 

$ 

34,000 

Hitachi 

$ 

55,000 

Army  Research 

Office 

$ 

140,000 

Hu^es  Aircraft 

$ 

10,000 

PUBLICATIONS 


Books: 


Pattern  Classifiers  and  Trainable  Machines  (with  G.  Wassel),  Springer- Verlag,  1981 


Books  edited: 

1.  Biomedical  Images  and  Computers  (co-edited  with  J.  C.  Bisconte),  Springer-Verlag, 
Heidelberg,  1982. 

2.  Proceedings  of  the  Symposium  on  Computer-Aided  Diafftosis  of  Medical  Images 
(editor),  IEEE  Computer  Society,  November  1976. 

3.  Pattern  Recognition:  Introduction  and  Foundations  (editor),  Dowden,  Hutchinson,  and 
Ross,  Stroudsburg,  PA,  1973. 


Book  Chapters  and  Articles: 

1 .  "Large-Scale  Feature  Selection"  (with  W.  Siedlecki),  Handbook  of  Pattern  Recognition 
&  Computer  Vision,  edited  by  C.H.  Chen,  LF.  Pau,  P.S.P  Wang,  World  Scientific, 
1993,  Chapter  1.3,  pp.  61-123. 

2.  "Biomedical  Image  Analysis"  (with  P.  V.  Sankar  and  R.  J.  Walter),  Handbook  of 
Pattern  Recognition  and  Image  Processing,  ed.  by  Young  and  Fu,  Chapter  26, 
Academic  Press,  1986. 

3.  "Two-Resolution  Detection  of  Lung  Tumors  in  Chest  Radiographs,"  (with  D. 
Petkovic),  Multiresolution  Image  Processing,  edited  by  A.  Rosenfeld,  Springer-Verlag, 
1983,  pp.  365-378.1 

4.  "The  Use  of  Markov  Random  Fields  as  Models  of  Texture"  (with  Martin  Hassner), 
Image  Modeling,  edited  by  A.  Rosenfeld,  Academic  Press,  1980,  pp.  185-198.  Also: 
Computer  Graphics  and  Image  Processing,  Vol.  12, 1980,  pp.  357-370. 

5.  "Toward  Computed  Detection  of  Nodules  in  Chest  Radiographs,"  Real  Time  Medical 
Image  Processing,  edited  by  M.  Onoe,  K.  Preston,  Jr.,  A.  Rosenfeld,  Plenum  Press, 
New  York,  1980,  pp.  53-59. 


6.  "Biomedical  Image  Analysis,"  Issues  in  Digital  Image  Processing,  edited  by  R.  M. 
Haralick  and  J.  C.  Simon,  published  by  Sijthoff  and  Noordhoff,  Alphen,  The 
Netherlands,  1980,  pp.  291-311. 

7.  "Image  Segmentation  and  Feature  Extraction,"  Digital  Image  Processing  and  Analysis, 
edited  by  J.  C.  Simon  and  A.  Rosenfeld,  NoordnofF  International  Publishers, 
Groningen,  The  Netherlands,  1977. 

8.  "Boundary  Detection  in  Medical  Radiographs,"  Digital  Processing  of  Biomedical 
Images,  edited  by  K.  Preston  and  M.  Onoe,  Plenum  Press,  New  York,  1976,  pp. 
309-322. 

9.  "Time-Varying  Threshold  Learning  "  Proceedings  of  1966  Joint  Automatic  Control 
Conference,  University  of  Washington,  Seattle,  Washington,  August,  1966,  pp.  33-37. 
Also:  Cybernetic  Problems  in  Bionics,  ed.  by  H.L.  Oestreicher  and  D.  R.  More,  Gordon 
and  Breach  Science  Publishers,  New  York,  1968,  pp.  687-695. 

Refereed  journal  articles: 

1.  J.  Sklansky,  Eric  Y.  Tao,  M.  Bazargan,  C.J.  Omes,  R.C.  Murchison,  S.  Teklehaimanot, 
“Computer-Aided,  Case-Based  Diagnosis  of  Mammographic  Regions  of  Interest  Containing 
Microcalcifications,”  Academic  Radiology,  Vol.  7,  No.  6,  June  2000,  pp.  395-405. 

2.  Kenneth  R.  Hoffmann  Ph.D. ,  Andreas  Wahle  Ph.D. ,  Claire  Pellot-Parakat  PhD.,  Jack  Sklansky 
Eng.Sc.D.,  Milan  Sonka,  PhD.,  “Biplane  Xray  Angiograms,  Intravascular  Ultrasound,  and  3D 
Visualization  of  Coronary  Vessels,”  submitted  to  International  Journal  of  Cardiac  Imaging,  July 
1999. 

3.  C.  Omes,  J.  Sklansky,  “A  Visual  Neural  Classifier,”  IEEE  Trans,  on  Systems,  Man,  and 
Cybernetics,  Part  B,  Vol.  28,  No.  4,  August  1998,  pp.  620-625. 

4.  Chester  Omes,  Jack  Sklansky,  “A  Neural  Network  that  Visualizes  What  It  Classifies,”  Pattern 
Recognition  Letters,  Vol.  18,  1997,  pp.  1301-1306. 

5.  B-B  Chai,  T.  Huang,  X.  Zhuang,  Y.  Zhao,  J.  Sklansky,  “Piecewise  linear  classifiers  using  binary 
tree  structure  and  genetic  algorithm,”  Pafferw  Recognition,  Vol.  29  No.  1 1, 1996,  pp.  1905-1917. 

6.  “  Genetic  Selection  and  Neural  Modeling  of  Piecewise  Linear  Classifiers”  (with  M.  Vriesenga), 
International  Journal  of  Artificial  Intelligence  and  Pattern  Recogyiition,  August  1996. 

7.  “Colored  Illumination  for  Enhancing  Discriminability  in  Machine  Vision”  (with  M.  Vriesenga, 

G.  Healey),  Journal  of  Visual  Communication  and  Image  Representation,  Vol.  6,  No.  3, 
September  1995,  pp.  244-255. 


8.  “Linear  Classifiers  by  Window  Training”  (with  L.  Bobrowski),  IEEE  Transactions  on  Systems, 
Man  and  Cybernetics,  Vol.  25,  No.  1,  Jan.  1995,  pp.  1-9. 

9.  J.  Sklansky,  “Toward  Quantitative  Coronary  Angiography,”  Proceedings  of  the  Japanese 
Association  of  Medical  Imaging  Technology,  Vol.  12,  No.  4,  July  1994,  pp.  295-305. 

10.  “Reconstructing  the  3-D  Medial  Axes  of  Coronary  Arteries  in  Single-View  Cineangiograms” 
(with  T.  V.  Nguyen),  IEEE  Transactions  on  Medical  Imaging,  Vol.  13,  No.  1,  March  1994,  pp 
61-73. 

11.  “Interftame  Interpolation  of  Cinematic  Sequences”  (with  J.  Ribas-Corbera),  Journal  of  Visual 
Communication  and  Image  Representation,  Vol.  4,  No.  4,  December  1993,  pp.  392-406. 

12.  "Graph-based  Thinning  for  Binary  Images"  (with  S.  Suzuki,  N.  Veda),  International  Journal  of 
Pattern  Recognition  and  Artificial  Intelligence,  Special  Issue  on  Techniques  for  Thinning 
Digitized  Patterns,  1993,  pp.  1009-1030. 

13.  L.V.  Tran,  R.C.  Bahn,  J.  Sklansky,  "Reconstructing  the  Cross  Sections  of  Coronary  Arteries  from 
Biplane  Angiograms",  IEEE  Trans,  on  Medical  Imaging,  Vol.  11,  No.  4,  Dec.  1992,  pp.  517- 
529. 

14.  "Flexible  Mask  Subtraction  for  Digital  Angiography"  (with  L.V.  Tran),  IEEE  Trans,  on  Medical 
Imaging,  Vol.  1 1,  No.  3,  September  1992,  pp.  407415. 

15.  "Robust  Classifiers  by  Mixed  Adaptation"  (with  D.  Gutfinger),  IEEE  Transactions  on  Pattern 
Analysis  and  Machine  Intelligence,  Vol.  13,  No.  6,  June  1991,  pp.  552-567. 

16.  "Automated  Design  of  Linear  Tree  Classifiers"  (with  Y.  Parl^,  Pattern  Recognition,  Vol.  23,  No. 
12, 1990,  pp.  1393-1412. 

17.  "A  note  on  genetic  algorithms  for  large-scale  feature  selection"  (with  W.  Siedlecki)  Pattern 
Recognition  Letters,  1989,  pp.  335-347. 

18.  "Macropipelined  multicomputer  architecture  for  image  analysis"  (with  Y.  Moon,  N. 
Bagherzadeh),  Journal  of  the  Optical  Society  of  American  A,  1989. 

19.  "Automated  design  of  multiple-class  piecewise  linear  classifiers  "  (with  Y.  Park),  Journal  of 
Classification,  1989,  pp.  195-222. 

20.  "Mapping  techniques  for  exploratory  pattern  analysis,"  (with  W.  Siedlecki  and  K.  Siedlecka), 
Pattern  Recognition  and  Artificial  Intelligence,  edited  by  E.S.  Gelsema  and  L.N.  Kanal,  Elsevier 
Science  Publishers  B.  V.  (North-Holland),  1988,  pp.  277-299. 


21.  "Bottlenecks  to  effective  application  of  machine  vision  -  a  discussion,"  Machine  Vision: 
Algorithms,  Architectures,  and  Systems,  edited  by  H.  Freeman,  Academic  Press,  1988,  pp.  187- 
192. 

22.  "Estimating  the  3-D  skeletons  and  transverse  areas  of  coronary  arteries  jfrom  biplane 
angiograms,"  (with  K.  Kitamura),  IEEE  Transactions  on  Medical  Imaging,  September  1988,  pp. 
173-187. 

23.  "On  automatic  feature  selection,"  (with  W.  Siedleclqj,  International  Journal  on  Pattern 
Recognition  and  Artificial  Intelligence,  Vol.  2,  No.  2,  August  1988,  pp.  197-220. 

24.  "Overview  of  mapping  techniques  for  exploratory  pattern  analysis,"  (with  W.  Siedlecki,  K 
Siedlecka),  Pattern  Recognition,  Vol.  21,  No.5,  pp.  431-438, 1988. 

25.  "Experiments  on  mapping  techniques  for  exploratory  pattern  analysis"  (with  W.  Siedlecki,  K. 
Siedlecka),  Pattern  Recognition,  Vol.  21,  pp.  431-438, 1988. 

26.  "Real-time  scene  imderstanding  and  vision  automation  ~  a  brief  overview,"  (with  K.  H.  Kim),  ed. 

by  Francis  J.  Corbett,  Proceedings  of  SPIE  —  The  International  Society  for  Optical  Engineering, 
Vol.  755,  Image  Pattern  Recognition  Algorithm  Implementations,  Techniques,  and  Technolo©^, 
January  1987,  pp.  126-131. 

27.  "Recursive  algorithms  for  implementing  digital  image  filters,"  (vvith  L.  Ferrari,  P.  V.  Sankar,  S. 
.Shinnaka),  IEEE  Transactions  on  Pattern  Analysis  and  Machine  Intelligence,  Vol.  PAMI-9,  No. 
3,  May  1987,  pp.  461-466. 

28.  "Feature  selection  for  automatic  classification  of  non-Gaussian  data"  (with  I.  Foroutan),  IEEE 
Trans,  on  Systems,  Man,  and  Cybernetics,  Vol.  SMC-17,  No.  3,  March/April  1987,  pp.  187-198. 

29.  "Multiple-Order  Derivatives  for  Detecting  Local  Image  Characteristics,"  (with  M.  Hashimoto), 

Computer  Vision,  Graphics,  and  Image  Processing,  Vol.  39,  1987,  pp.  28-55. 

30.  "Computing  the  Skeleton  of  Coronary  Arteries  in  Cineangiograms,"  (with  T.  Nguyen), 
Computers  and  Biomedical  Research,  October  1986,  pp.  428-444. 

31.  "Efficient  two-dimensional  filters  using  B-spline  fimctions,"  (with  L.  Ferrari,  P.  V.  Sankar  and  S. 
Leeman),  Computer  Graphics  and  Image  Processing,  Vol.  35, 152-169, 1986. 

32.  "Biomedical  Image  Analysis,"  (with  P.  V.  Sankar  and  R.  J.  Waited,  Handbook  of  Pattern 
Recognition  and  Image  Processing  edited  by  Young  and  K-S  Fu,  Chapter  26,  Academic  Press, 
1986,  pp.  629-647. 

33.  "Elimination  of  nonpivotal  plane  images  fix)m  x-ray  motion-  tomograms,"(with  S.  Kawata), 
IEEE  Transactions  on  Medical  Imaging,  Vol.  Mi-4,  No.  3,  September  1985. 


34.  "The  relative  nei^borhood  graph  for  mixed  feature  variables,"  (with  M.  Ichino^,  Pattern 
Recognition,  Vol.  8,  No.  2, 1985,  pp.  161-167. 

35.  "A  note  on  Duhamel  Integrals  and  Running  Average  Filters,"  (with  LFerrari),  Computer  Vision 
Graphics,  and  Image  Processing,  Vol.  29,  pp.  358-360, 1985. 

36.  "Optimum  feature  selection  by  zero-one  integer  programming,"  (with  M.  Ichinql,  IEEE  Trans,  on 
Systems,  Man,  and  Cybernetics,  Vol.  14,  October  1984,  pp.  737-746. 

37.  "A  fast  recursive  algorithm  for  binary  balued  two-dimensional  filters"  (with  L.  Ferrari), 
Computer  Vision,  Graphics  and  Image  Processing,  Vol.  26, 292-302, 1984. 

38.  "Minimal  Rectangular  Partitions  of  Digitized  Blobs,"  Cow/TM/er  Vision,  Graphics,  and  Image 
Processing,  (with  L  Ferrari,  P.V.  Sankar),  Vol.  28,  pp.  58-71, 1984. 

39.  "Circularly  coded  x-ray  tomography,"  (with  S.  Kawata),  Transactions  of  Applied  Optics,  Vol.  23, 
No.  10,Mayl5,  1984. 

40.  "Scan-along  polygonal  approximation  for  data  compression  of  electrocardiograms,"  (with  M. 
Ishijima.  S.  Shin,  G.  Hostette^,  IEEE  Trans,  on  Biomedical  Engineering,  November  1983,  pp. 
723-729. 

41.  "Two-resolution  detection  of  lung  tumors  in  chest  radiographs,"  (with  D.  Petkovic), 
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Rationale  and  Objectives.  The  purpose  of  this  study  was  to  evaluate  the  effectiveness  of  a  mapped-database  diagnostic  sys¬ 
tem  in  reducing  the  incidence  of  benign  biopsies  and  misdiagnosed  cancers  among  mammographic  regions  of  interest  (ROIs). 

Materials  and  Methods.  A  novel  neural  network  was  devised  {a)  to  respond  to  a  query  ROI  by  recommending  to  biopsy  or 
not  to  biopsy  and  {h)  to  map  each  ROI  in  the  database  as  a  dot  on  a  computer  screen.  The  network  was  designed  so  that  clus¬ 
ters  in  the  array  of  dots  help  the  radiologist  to  find  proved  ROIs  visually  similar  to  the  query  ROI.  This  mapped-database  diag¬ 
nostic  system  was  restricted  to  ROIs  with  visible  microcalcifications.  The  neural  network  was  trained  with  a  stored  database  of 
80  biopsy-proved  ROIs. 

Results.  Four  radiologists  acting  independently  on  100  ROIs  recommended  biopsies  for  18,  15,  28,  and  18  benign  ROIs  and 
misdiagnosed  cancers  in  11,  12,  7,  and  eight  ROIs,  respectively.  Interaction  with  the  mapped-database  system  reduced  the 
numbers  of  benign  biopsies  to  11,  eight,  18,  and  10  cases  and  of  misdiagnosed  cancers  to  eight,  seven,  four,  and  three  cases, 
respectively.  Statistical  analysis  indicated  that  three  radiologists  achieved  significant  improvements  at  P  <  .02  and  the  fourth 
achieved  a  substantial  improvement  at  P  <  .07. 

Conclusion.  By  using  a  mapped  database  of  proved  mammographic  ROIs  containing  microcalcifications,  radiologists  may 
statistically  significantly  reduce  the  numbers  of  benign  biopsies  and  misdiagnosed  cancers. 

Key  Words.  Computer-aided  diagnosis;  digital  mammography;  microcalcifications. 


A  critical  aspect  of  mammographic  diagnosis  is  deciding 
whether  to  recommend  biopsy.  Approximately  two  benign 
lesions  are  sampled  for  biopsy  for  every  malignant  lesion 
detected  (1).  In  a  recent  study  of  2,400  women  undergoing 
mammographic  screening  during  a  10-year  period  (2),  an 
additional  $33  was  spent  on  evaluating  false-positive  re- 
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suits  for  every  $100  spent  on  screening.  Among  the  women 
in  that  study,  the  cumulative  risk  of  a  false-positive  result 
after  10  mammograms  was  49.1%.  Thus,  it  would  be  highly 
desirable  to  reduce  the  frequency  of  benign  biopsies  during 
mammographic  screening  without  increasing  the  number  of 
missed  cancers. 

The  objective  of  this  study  was  to  evaluate  the  effective¬ 
ness  of  a  recently  devised  “mapped-database  diagnostic 
system”  in  reducing  the  frequency  of  benign  biopsies  and 
misdiagnosed  cancers  among  mammographic  regions  of 
interest  (ROIs).  The  principal  difference  between  this  sys¬ 
tem  and  earlier  computer-aided  diagnostic  systems  for  mam¬ 
mography  is  the  facilitation  of  diagnostic  reasoning  by  a 
database  of  proved  ROIs.  In  this  study,  the  presented  mam¬ 
mographic  images  and  stored  database  are  restricted  to  ROIs, 
and  the  ROIs  themselves  are  restricted  to  those  revealing 
apparent  microcalcifications. 

Interpretation  of  mammograms  consists  of  two  major 
components:  detection  and  diagnosis.  In  detection,  the 
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radiologist  typically  examines  four  full-field  views — two 
views  of  each  breast.  In  these  views,  the  radiologist  may 
detect  one  or  more  ROIs  revealing  possible  lesions.  In  diag¬ 
nosis,  the  radiologist  analyzes  one  or  more  ROIs  to  deter¬ 
mine  whether  biopsy  should  be  performed  or  whether  the 
patient  should  be  recalled  for  further  examination.  Thus, 
analysis  of  ROIs  is  an  important  part  of  clinical  mammogra¬ 
phy,  particularly  in  establishing  a  diagnosis.  This  study  fo¬ 
cused  on  the  ability  of  the  mapped-database  diagnostic  sys¬ 
tem  to  reduce  the  number  of  negative  biopsy  results  and  of 
misdiagnosed  cancers  associated  with  the  analysis  of  ROIs. 

By  restricting  the  images  used  in  this  study  to  ROIs, 
thereby  excluding  the  full-breast  views,  we  reduced  much 
of  the  cost  and  duration  of  the  experiments,  but  at  the  ex¬ 
pense  of  eliminating  contextual  evidence.  A  further  reduc¬ 
tion  in  cost — and  in  diagnostic  accuracy — was  obtained  by 
replacing  film  mammograms  with  digital  images  displayed 
with  a  cathode  ray  tube  monitor  (3).  By  anticipating  the  wide¬ 
spread  growth  of  digital  mammography  while  recognizing 
these  reductions  in  diagnostic  accuracy,  this  relatively  inex¬ 
pensive  study  examined  the  effect  of  the  mapped-database 
system  on  diagnostic  accuracy  during  digital  mammography. 

In  the  mapped-database  system,  an  artificial  neural  net¬ 
work  responds  to  a  mammographic  ROI  by  recommending, 
or  not  recommending,  biopsy.  In  addition,  the  neural  network 
maps  a  database  of  ROIs  from  proved  mammograms  onto  a 
two-dimensional  display  as  an  aid  to  establishing  a  diagnosis. 
The  network  does  this  by  mapping  each  ROI  into  a  dot  on  a 
screen  and  arranging  these  dots  so  that  they  cluster  into  radio¬ 
graphically  similar  subclasses.  We  refer  to  this  representation 
as  a  mapped  database.  In  addition  to  producing  the  mapped 
database,  the  neural  network  partitions  the  screen  into  two  de¬ 
cision  regions:  one  for  a  biopsy  being  recommended,  and  the 
other  for  a  biopsy  not  being  recommended.  We  call  the  result¬ 
ing  display  a  relational  map  (4).  By  annexing  this  neural  net¬ 
work  to  a  means  of  quickly  retrieving  and  viewing  ROIs  that 
are  mapped  close  to  the  query,  the  radiologist  is  given  a 
simple  way  of  conversing  with  the  database  to  enhance  the 
reliability  of  the  diagnostic  decision.  Thus,  this  system  ampli¬ 
fies  the  radiologist’s  powers  of  “case-based  reasoning”  (5) 
and  acts  somewhat  like  a  second  reader  with  consensus  (6). 

This  concept  led  to  construction  of  a  mammographic  di¬ 
agnostic  system  built  on  the  basis  of  several  earlier  devel¬ 
opments:  {a)  algorithms  for  detecting  and  analyzing  clus¬ 
ters  of  microcalcifications  (7-10),  {b)  large  databases  of 
digitized  mammograms  (11-15),  (c)  genetic  algorithms  for 
high-speed  parallel  search  (16),  and  {d)  “visual  neural  clas¬ 
sifiers”  (ie,  specialized  neural  networks)  that  map  multidi¬ 
mensional  features  and  multidimensional  decision  surfaces 


onto  two-dimensional  displays  (4).  We  refer  to  this  as  a 
mapped-database  diagnostic  system.  The  test  results  re¬ 
ported  herein  suggest  that  for  mammograms  with  detectable 
microcalcifications,  this  system  can  provide  a  new  level  of 
reliability  for  mammographic  diagnosis. 

The  most  pertinent  earlier  work  regarding  database-aided 
mammographic  diagnosis  is  that  by  Swett  and  his  colleagues 
at  Yale  University  (17).  Their  mammographic  database  sys¬ 
tem,  MAMMO/ICON,  required  a  verbal  medical  history  and 
verbal  mammographic  descriptors  to  supplement  each  mam¬ 
mogram.  This  system  could  provide  diagnostic  advice  that 
depended  on  semantic  similarities  of  the  verbal  descriptors 
and  medical  histories  (18).  A  shortcoming  of  this  technology, 
however,  was  its  dependence  on  verbal  descriptors  to  match 
information  that  was  essentially  pictorial  and  nonverbal. 
Among  radiologists,  there  may  be  considerable  variability  in 
the  specific  words  used  to  describe  the  pertinent  properties 
of  any  given  mammogram.  This  variability  may  impair  the 
reliability  of  retrieving  similar  mammograms  filed  according 
to  verbal  descriptors.  Another  shortcoming  was  the  manual 
construction  of  the  verbal  descriptors  for  the  mammograms, 
in  contrast  to  the  possibility  of  automatic  extraction  and 
counting  of  key  words  in  text. 

Among  other  published  articles  on  computer-aided  diag¬ 
nosis  of  mammograms,  those  by  Getty  et  al  (19),  Jiang  et  al 
(9),  and  Chan  et  al  (20)  are  particularly  relevant  to  the  pres¬ 
ent  work.  Getty  et  al  (19)  demonstrated  that  a  checklist  of 
descriptors  (ie,  “features”)  combined  with  a  digital  classifier 
could  be  an  effective  aid  in  establishing  the  diagnosis  during 
mammography.  These  features,  as  in  our  experiment,  were 
for  the  most  part  descriptors  of  a  “focused  abnormality”  (ie, 
the  equivalent  of  ROI  as  used  here).  The  protocol  for  aiding 
the  radiologist  included  a  computer-alone  reading,  which  was 
followed  with  a  second  reading  by  the  radiologist  (analogous 
to  a  double  reading  with  consensus).  A  second  contribution 
by  Getty  et  al  was  the  demonstration  that  a  carefully  selected 
set  of  features,  when  coupled  with  a  session  to  train  radiolo¬ 
gists  to  estimate  these  features,  could  enable  computer-aided 
generalists  to  read  film  mammograms  almost  as  well  as 
highly  skilled  specialists.  Jiang  et  al  (9)  demonstrated  the 
potential  of  automatically  extracting  features  from  microcal¬ 
cifications,  coupled  with  an  artificial  neural  network,  for 
computer-aided  diagnosis.  The  methods  in  both  of  these 
studies  (9,19),  however,  still  required  substantial  human  in¬ 
volvement  in  the  computer-aided  procedure.  In  the  method 
of  Getty  et  al  (19),  each  radiologist  estimated  the  numeric 
level  of  intensity  or  level  of  confidence  for  the  existence  of 
each  feature;  in  the  method  of  Jiang  et  al  (9),  the  location  of 
each  microcalcification  was  determined  manually.  This 
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amount  of  human  involvement  likely  makes  these  techniques 
impractical  for  clinical  use. 

Chan  et  al  (20)  restricted  human  involvement  to  identi¬ 
fying  in  each  view  an  ROI  enclosing  a  mass.  This  level  of 
human  involvement  is  likely  to  be  practical  during  clinical 
applications,  because  finding  ROIs  is  part  of  the  usual  pro¬ 
cedure  for  interpreting  mammograms.  As  in  the  method  of 
Getty  et  al  (19),  the  radiologist-reader  in  the  method  of  Chan 
et  al  (20)  was  asked  to  make  a  final  estimate  regarding  the 
likelihood  of  malignancy  in  a  presented  mammogram  after 
first  performing  an  unaided  estimation  and  then  receiving 
the  computer’s  estimate.  In  that  study,  the  performance  of 
the  computer  alone  was  comparable  to  that  of  the  best  per¬ 
forming  unaided  radiologist-reader  and  to  the  performance 
of  six  aided  radiologist-readers  as  a  group. 


MATERIALS  AND  METHODS 


A  retrospective  study  of  the  mapped-database  system  was 
performed  during  June  1998.  The  ROIs  included  in  this 
study  were  restricted  to  those  exhibiting  microcalcifications. 
For  this  study,  we  acquired  a  database  of  200  ROIs  from  138 
proved  cases.  Biopsy  results  were  included  with  each  ROI. 
Four  radiologists  from  the  clinical  faculty  of  the  Department 
of  Radiology  at  King/Drew  Medical  Center  (KDMC)  read 
the  digitized  images  in  this  study.  None  of  these  radiologists 
was  familiar  with  the  cases,  and  all  of  them  were  certified  for 
mammography  by  the  U.S.  Food  and  Drug  Administration. 
Their  postresidency  experience  in  reading  mammograms 
was  29  years,  10  years,  5  years,  and  6  months;  these  readers 
are  referred  to  as  Rj,  R2,  R3,  and  R^,  respectively. 

The  ROIs  in  the  database  were  excised  from  digitizations 
of  film  mammograms  provided  by  KDMC  and  by  the  Univer¬ 
sity  of  California  at  Los  Angeles  (UCLA)  Iris  Cantor  Center 
for  Breast  Imaging.  In  all  cases  associated  with  these  mammo¬ 
grams,  either  biopsy  or  follow-up  with  subsequent  mammog¬ 
raphy  was  performed.  From  these  mammograms,  all  138  cases 
that  revealed  microcalcifications  were  selected  for  this  study. 
Fifty-two  cases  were  from  KDMC,  and  86  were  from  UCLA. 
The  selected  mammograms  were  digitized  at  KDMC  on  a 
scanner-digitizer  (Lumiscan  85;  Lumisys,  Sunnyvale,  Calif) 
at  a  pixel  width  of  50  (im  and  a  pixel  depth  of  12  pm. 

For  each  film  mammogram,  the  location  of  an  abnor¬ 
mality  containing  a  cluster  of  microcalcifications  was  pro¬ 
vided  in  the  medical  record  that  accompanied  the  image. 
From  this  information,  we  constructed  a  512  x  512-pixel 
ROI  enclosing  each  designated  cluster  of  microcalcifica- 
tions.  This  construction  yielded  160  ROIs.  No  two  ROIs  in 
this  set  were  views  of  the  same  lesion.  To  each  ROI  we  ap¬ 


plied  an  automatic  microcalcification  detector  and  segmenter 
that  was  developed  during  an  earlier  study  (7).  An  additional 
40  normal  ROIs  were  also  added  to  this  database,  thereby 
making  a  total  of  200  ROIs  in  this  study.  These  normal  ROIs 
were  included  to  ensure  that  readers  would  have  the  impres¬ 
sion  that  not  all  ROIs  in  the  set  used  to  test  the  system  (de¬ 
scribed  later)  were  abnormal.  The  normal  ROIs  were  ob¬ 
tained  from  cases  involving  patients  who  did  not  undergo 
biopsy  and  were  not  recalled  during  a  period  of  at  least  18 
months.  Within  each  of  these  cases,  the  normal  ROI  was 
selected  arbitrarily. 

Of  the  abnormal  ROIs  (ie,  those  containing  microcalcifi¬ 
cations),  64  were  proved  at  biopsy  to  be  benign,  49  were 
proved  at  biopsy  to  be  malignant,  and  47  were  proved  at  fol¬ 
low-up  to  be  benign  (ie,  both  the  ROI  and  the  follow-up 
findings  did  not  produce  a  recommendation  for  biopsy).  At 
least  18  months  elapsed  between  two  successive  examina¬ 
tions,  and  none  of  the  radiologists  on  our  reading  panel  had 
seen  these  ROIs  before  this  study.  One  radiologist  (R,)  parti¬ 
tioned  the  200  ROIs  into  groups  such  that  ROIs  in  the  same 
group  were  visually  similar.  We  refer  to  these  groups  as  per¬ 
ceptual  groups.  No  restriction  was  placed  on  the  number  of 
perceptual  groups,  although  the  radiologist  (Rj)  was  in¬ 
formed  that  eight  groups  were  formed  by  another  radiologist 
during  another  set  of  ROIs  in  an  earlier  experiment  (21).  The 
radiologist  (Rj)  partitioned  the  200  ROIs  into  the  following 
12  groups:  {a)  amorphous  with  mass,  {h)  lobular  and  ductal, 
(c)  amorphous,  {d)  pleomorphic  and  scattered,  {e)  lobular, 
(/)  granular,  (g)  casting,  {h)  punctate  scattered,  (i)  linear  duc¬ 
tal,  (j)  vesicular,  (k)  oil  cyst,  and  (/)  no  visible  abnormality. 

Computer-aided  reading  of  the  test  set  by  the  reading 
panel  was  performed  more  than  a  month  after  partitioning  of 
the  ROIs  into  perceptual  groups.  The  intent  of  this  1 -month 
delay  was  to  suppress  the  memory  of  the  200  ROIs  in  the  ra¬ 
diologist  who  partitioned  them  into  the  perceptual  groups. 

The  database  was  then  divided  into  two  sets,  D  and  7,  each 
of  which  consisted  of  100  ROIs.  D  was  used  for  the  design  of 
the  diagnostic  system,  and  T  was  used  for  testing  of  the  sys¬ 
tem.  The  160  abnormal  ROIs  were  randomly  partitioned 
several  times  into  two  equal  parts,  and  7^,  until  a  (D^,  7^) 
pair  was  found  such  that  and  7^  each  contained  at  least  two 
ROIs  from  each  perceptual  group.  (Ensuring  representation 
of  at  least  two  ROIs  from  each  perceptual  group,  however, 
may  have  biased  the  diagnostic  system  somewhat  in  favor  of 
the  neural  network.  On  the  other  hand,  not  achieving  this  rep¬ 
resentation  would  have  biased  the  study  against  the  neural 
network.)  consisted  of  54  benign  and  26  malignant  ROIs. 
7^  consisted  of  57  benign  and  23  malignant  ROIs.  Because  no 
two  ROIs  were  images  of  the  same  lesion,  all  lesions  in  7 
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were  distinct  from  all  lesions  in  Z).  The  40  normal  ROIs  were  of  and  T^.  These  procedures  were  performed  to  ensure  the 

partitioned  several  times  into  two  equal  parts,  and  until  design  set  was  representative  of  the  types  of  abnormahties  oc- 

a  (Z)^,  pair  was  found  such  that  and  had  similar  dis-  curring  in  the  test  set.  As  mentioned,  the  normal  ROIs  were 
tributions  in  the  mapped  feature  space.  The  final  design  set  D  included  to  ensure  the  readers  would  have  the  impression  that 
was  the  union  of  and  The  final  test  set  T  was  the  union  not  all  ROIs  in  T  were  abnormal.  Because  diagnostic  (rather 
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Figure  1.  Numeric  labeling  for  the  pixels  of  a  digital  model  repre¬ 
senting  a  typical  microcalcification.  The  label  of  each  pixel  is  the 
number  of  pixel  displacements  of  that  pixel  from  the  medial  axis  of 
the  boundary  of  the  microcalcification.  Thus,  0  denotes  a  pixel  on 
the  medial  axis  of  the  boundary,  1  a  pixel  just  inside  the  boundary, 
-1  a  pixel  just  outside  the  boundary,  and  ~2  a  pixel  just  outside 
the  “1  region. 


than  detection)  efficacy  was  paramount  in  this  study,  the 
mapped-database  system  in  this  study  was  designed  to  ana¬ 
lyze  only  the  abnormal  ROIs.  Consequently,  readings  of  the 
normal  ROIs  were  not  included  in  our  calculations  of  speci¬ 
ficity,  sensitivity,  and  other  measures  of  performance. 

From  each  abnormal  ROI,  18  candidate  features  were  ex¬ 
tracted.  Each  candidate  feature  was  designed  to  reflect  one  or 
more  properties  of  calcifications  (or  groups  of  calcifications) 
that  are  meaningful  to  the  radiologist.  These  features  were 
based  on  the  American  College  of  Radiology's  Breast  Imag¬ 
ing  Recording  and  Data  System  (BI-RADS)  descriptors  for 
interpreting  mammograms,  the  book  by  Lanyi  (22),  and  dis¬ 
cussions  with  the  radiologists  on  our  panel.  Unfortunately, 
most  BI-RADS  descriptors  are  too  subjective  for  encoding 
into  computer-executable  formulas.  Consequently,  the  fea¬ 
tures  we  extracted  were  inspired  by,  but  not  necessarily  a  di¬ 
rect  implementation  of,  the  BI-RADS  descriptors.  A  formula¬ 
tion  and  brief  clinical  justification  for  each  of  these  features 
are  given  in  Table  1.  In  the  first  two  columns  of  this  table,  a 
name  and  a  mathematical  symbol  denote  each  feature.  A  for¬ 
mula  for  each  feature  is  given  in  the  third  column.  The  fourth 
column  contains  a  brief  description  of  the  clinical  importance 
of  the  feature.  Some  of  the  formulas  refer  to  numerical  pixel 


labels  -2,-1,  0,  and  1.  The  label  on  each  pixel  P  is  the  num¬ 
ber  of  pixel  displacements  of  P  from  the  boundary  of  a  calci¬ 
fication.  The  label  0  denotes  a  pixel  on  the  boundary^  1  de¬ 
notes  a  pixel  just  inside  the  boundary,  -1  denotes  a  pixel  just 
outside  the  boundary,  and  -2  denotes  a  pixel  just  outside  the 
-1  region.  These  labels  are  illustrated  in  Figure  1,  which 
shows  a  digital  model  of  a  typical  microcalcification. 

Starting  from  this  set  of  candidate  features,  a  genetic  al¬ 
gorithm  searched  for  those  subsets  of  features  that  were  most 
effective  in  discriminating  malignant  from  benign  cases  in 
the  database.  This  algorithm  was  designed  in  accordance 
with  the  genetic  feature  selector  described  by  Siedlecki  and 
Sklansky  (23).  In  this  algorithm,  each  candidate  subset  is 
represented  by  a  binary  string  (ie,  a  sequence  of  Os  and  Is). 
The  value  (0  or  1)  of  the  nth  element  of  the  string  indicates 
whether  the  nth  feature  belongs  to  the  candidate  subset.  We 
refer  to  this  string  as  a  chromosome,  A  population  of  100 
chromosomes  was  transformed  by  an  evolutionary  process 
consisting  of  mutation,  mating,  and  reproduction  to  form  a 
sequence  of  new  populations  or  “generations”  under  the 
guidance  of  a  penalty  function  that  accounted  for  the  /:-near- 
est-neighbor  discriminability  of  the  feature  vectors  in  each 
feature  subset  and  for  the  number  of  features  in  the  subset. 
The  sizes  of  the  new  populations  were  kept  constant  at  100. 
This  evolutionary  process  also  optimized  the  value  of  k.  To 
ensure  an  acceptably  low  rate  of  misdiagnosed  cancers,  the 
penalty  function  gave  seven  times  as  much  weight  to  misdi¬ 
agnosing  a  cancer  as  to  recommending  a  biopsy  that  was 
benign.  (The  weight  was  chosen  empirically  so  that  the 
nearest-neighbor  classifier  would  produce  only  one  or  two 
missed  cancers  on  the  data  set.)  This  algorithm  selected 
the  following  four  features:  {a)  mean  area,  {h)  mean  as¬ 
pect  ratio,  (c)  mean  irregularity  (ie,  noncircularity),  and 
(d)  number  of  microcalcifications.  Another  algorithm  ex¬ 
tracted  these  four  features  from  each  ROI,  thus  forming  a 
four-dimensional vector  as  a  descriptor  of  that  ROI. 

A  five-layer  neural  network  was  constructed  in  accor¬ 
dance  with  the  architecture  and  the  design  principles  de¬ 
scribed  by  Omes  and  Sklansky  (24).  Each  node  in  this  archi¬ 
tecture  represented  an  artificial  neuron  that  consisted  of  a 
weighted  summation  followed  by  a  sigmoidal  activation 
function.  The  first  layer  consisted  of  four  input  neurons,  one 
for  each  selected  feature.  The  second  and  third  layers  con¬ 
sisted  of  three  and  two  neurons,  respectively,  to  reduce  the 
dimensionality  of  feature  space.  The  two-neuron  layer  pro¬ 
duced  X  and  y  coordinates  (one  from  each  of  the  two  neu¬ 
rons)  to  represent  each  ROI  as  a  dot  on  a  relational  map.  For 
each  ROI  in  the  design  set,  this  dot  was  labeled  to  indicate 
whether  the  biopsy  was  benign  or  malignant.  The  two 
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neurons  in  the  fifth  layer  classified  each  feature  vector  as 
either  “biopsy  recommended”  or  “biopsy  not  recom¬ 
mended.”  (The  neuron  producing  the  larger  of  the  two  out¬ 
puts  determined  the  class  of  the  feature  vector.)  By  entering 
every  permissible  pair  of  map  coordinates  at  the  input  to  the 
fourth  layer,  the  fifth  layer  thus  produced  a  relational  map 
in  which  all  pixels  were  labeled  as  either  biopsy  recom¬ 
mended  or  biopsy  not  recommended. 

Using  the  four  features  selected  by  the  genetic  algorithm, 
the  neural  network  was  trained  by  backpropagation  (ie,  a 
form  of  gradient  descent  in  the  space  formed  by  the  weights 
at  the  inputs  to  the  neurons)  on  the  design  set  D^,  thereby  pro¬ 
ducing  the  trained  neural  network  NN(Z)^)  (4).  In  accordance 
with  our  design  philosophy  for  this  study,  the  neural  network 
was  trained  only  on  abnormal  ROIs.  The  biopsy  outcomes 
(ie,  benign  and  malignant)  in  the  database  were  the  desired 
output  classes  in  the  trained  neural  network,  which  produced 
a  two-dimensional  map  of  and  a  near-optimal  partition  of 
this  map  into  two  decision  regions.  These  decision  regions 
were  associated  with  the  decisions  of  biopsy  recommended 
and  biopsy  not  recommended.  The  training  was  stopped 
when  the  mean  square  error  converged  to  a  nearly  constant 
value.  This  design  strategy,  and  the  relatively  low  error  rate  of 
the  neural  network  on  the  test  set  7,  encouraged  us  to  believe 
that  the  network  was  then  adequately  trained. 

The  boundary  separating  the  decision  regions  is  referred 
to  as  a  decision  curve.  The  decision  curve  is  substantially 
smoother  than  the  variability  of  the  data,  which  is  the  result 
of  the  averaging  produced  by  the  backpropagation  training 
and  the  small  number  of  neurons  compared  with  the  number 
of  feature  vectors  in  the  training  set.  The  neural  network 
tends  to  produce  decision  regions  that  are  responsive  to  sta¬ 
tistical  models  of  the  training  data  rather  than  to  the  training 
data  itself.  Specifically,  the  training  algorithm  minimized  a 
weighted  sum-of-squared-error  function,  which  yielded  out¬ 
puts  that  were  estimates  of  Bayesian  posterior  probabilities 
that  the  input  vector  belonged  to  the  corresponding  class  (ie, 
biopsy  recommended  or  biopsy  not  recommended).  Thus, 
the  decision  curve  produced  by  the  neural  network  tends  to 
“generalize”  the  training  data  and,  thereby,  to  make  good  de¬ 
cisions  on  future  data  not  included  in  the  training  set  (25). 

The  trained  neural  network  displayed  the  query  and  the 
ROIs  in  as  dots,  with  each  dot  being  located  by  the  x  and 
y  coordinates  produced  by  the  two  neurons  (one  coordinate 
from  each  neuron)  at  the  two-neuron  layer  of  the  network. 

We  refer  to  the  space  spanned  by  x  and  y  as  the  mapped  fea¬ 
ture  space  and  to  the  map  of  as  a  mapped  database.  The 
map  of  and  its  decision  regions  are  depicted  in  Figure  2. 
We  call  this  a  relational  map.  Here,  the  black  region  repre- 


Figure  2.  Relational  map  of  design  set  D  and  the  query  ROI.  Each 
malignant  case  is  labeled  X,  and  each  benign  case  is  labeled  O. 
The  query  ROI  is  labeled  ■. 


sents  biopsy  recommended,  and  the  white  region  represents 
biopsy  not  recommended.  The  decision  curve  is  the  bound¬ 
ary  between  the  black  and  the  white  regions.  The  symbols  O 
and  X  denote  benign  and  malignant  feature  vectors,  respec¬ 
tively.  The  symbol  ■  denotes  a  query. 

A  weakness  of  the  mapped-database  system  is  that  it  does 
not  reveal  the  quantitative  relationships  between  the  mapped 
feature  space  and  the  unmapped  features.  This  weakness  is 
compensated  for,  however,  by  the  ability  of  the  relational 
map  to  cluster  visually  and  diagnostically  similar  ROIs  and 
by  the  map’s  representation  of  a  multidimensional  decision 
surface  as  a  decision  curve  (or  as  several  disjoint  decision 
curves)  in  the  mapped  feature  space.  The  clustering  helps  the 
radiologist  to  find  ROIs  that  are  visually  similar  to  the  query, 
and  the  decision  curve  helps  the  radiologist  to  determine  the 
confidence  of  the  neural  network  in  its  recommendations  ei¬ 
ther  for  or  against  performing  biopsy.  Further  insight  into 
the  relationships  between  the  mapped  feature  space  and  the 
original  features  can  be  obtained  by  making  available  the  nu¬ 
merical  values  of  the  original  features  of  each  mapped  ROI 
to  the  radiologist-user. 

The  user  interface  presented  on  the  computer  monitor  is  il¬ 
lustrated  in  Figure  3.  This  interface  guides  the  radiologist-user 
through  the  steps  of  diagnosis.  Identification  numbers  in  this 
figure  (eg,  011318)  were  constructed  so  that  patients  could  not 
be  identified  from  the  information  in  the  figure.  The  mammo- 
graphic  image  in  the  upper  left  is  an  enlargement  of  the  ROI 
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Figure  3.  User  Interface  for  mammographic  diagnosis. 


of  a  query  mammogram.  Digital  magnification  and  negative- 
to-positive  inversion  were  provided  as  options  on  the  user  in¬ 
terface.  In  response  to  a  prompt  from  the  interface,  the  radi¬ 
ologist  first  determines  an  “unaided”  diagnosis  of  this  ROI  and 
reports  it  in  the  BI-RADS  code  (26)  by  clicking  N,  B,  P,  S,  or 
M,  which  denote  normal,  benign,  probably  benign,  suspicious 
(possibly  malignant),  and  almost  definitely  malignant,  respec¬ 
tively.  In  this  study,  we  assumed  that  N,  B,  and  P  corresponded 
to  biopsy  not  recommended  and  that  S  and  M  corresponded  to 
biopsy  recommended.  This  assumption  conformed  to  the  clini¬ 
cal  practice  at  KDMC.  We  instructed  each  reader  to  produce 
the  unaided  diagnosis  as  if  it  were  the  final  diagnosis  for  the 
examination.  No  time  limit  was  imposed  on  this  diagnosis. 

In  this  study,  the  diagnostic  process  began  by  establish¬ 
ing  an  unaided  diagnosis,  which  was  followed  immediately 
by  establishing  an  aided  diagnosis  (analogous  to  double 
reading).  We  assumed  that  the  operating  point  (ie,  the  sub¬ 
jective  decision  threshold  for  a  BI-RADS  symbol)  of  the 
unaided  radiologist  might  be  affected  by  that  radiologist’s 
anticipation  of  an  aided  diagnosis,  but  that  the  receiver  op¬ 
erating  characteristic  (ROC)  of  the  unaided  diagnosis  was 
unaffected  by  that  anticipation. 

Immediately  after  the  unaided  diagnosis  was  established, 
the  relational  map  was  presented  to  the  radiologist.  This  map 
contained  the  mapped  database,  mapped  query,  and  distinc¬ 
tively  colored  decision  regions.  Thumbnail  images  of  six 


ROIs  near  the  query  in  the  mapped  feature  space  were  pre¬ 
sented  at  the  right  of  the  monitor  screen.  From  these  images, 
the  radiologist  retrieved  for  review  those  with  a  visual  appear¬ 
ance  that  seemed  to  be  most  similar  to  that  of  the  query.  This 
review  was  facilitated  by  an  enlarged  view  of  the  retrieved 
ROI  and  the  accompanying  biopsy  reports.  On  the  basis  of 
this  interaction,  the  radiologist  selected  a  BI-RADS  score  for 
the  ROL  Again,  the  reader  was  not  restricted  by  a  time  limit 
for  establishing  this  diagnosis,  and  both  magnification  and  in¬ 
version  were  provided  as  options  on  the  user  interface. 

To  evaluate  the  diagnostic  improvement  provided  by  the 
mapped-database  diagnostic  system,  we  computed  the  ROC 
of  each  unaided  radiologist,  of  each  computer-aided  radiolo¬ 
gist,  and  of  the  computer  alone.  Each  ROC  accounted  for  the 
BI-RADS  responses  of  the  radiologists  and  for  the  outcomes 
of  the  biopsies.  The  diagnostic  performance  of  each  radiolo¬ 
gist  was  measured  by  the  area  under  the  ROC  curve,  which 
is  an  estimate  of  the  probability  for  a  correct  decision  in  a 
forced  choice  between  two  ROIs,  one  of  which  is  malignant 
and  the  other  of  which  is  benign  (27).  Thus,  A  must  lie  be¬ 
tween  a  minimum  of  0.5  (corresponding  to  an  unbiased  ran¬ 
dom  guess)  and  a  maximum  of  1  (perfect  performance). 

We  also  computed  the  average  performance  of  the  four  ra¬ 
diologists  when  unaided  by  the  mapped-database  system  and 
when  aided  by  the  mapped-database  system.  LABMRMC 
(C.E.  Metz,  LABMRMC  LOB,  beta  version  3,  University  of 
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Figure  4.  ROC  curves  for  radiologist  and  for  the  computer 
alone. 
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Figure  5.  ROC  curves  for  radiologist  R^  and  for  the  computer 
alone. 


Chicago,  Chicago,  Ill)  (28)  ^vas  employed  to  calculate  the 
maximum  likelihood  estimates  for  the  parameters  of  binor¬ 
mal  models  and  the  statistical  significance  of  the  change  in 
areas  under  the  ROC  curves  for  the  unaided  and  the  aided 
reading  modalities.  The  ROC  curves  representing  the  aver¬ 
age  performance  of  the  unaided  radiologists  were  computed 
by  averaging  their  estimated  slopes  and  intercepts  in  normal 
deviate  space.  This  same  process  was  also  performed  for  the 
aided  radiologists. 

The  mapped-database  diagnostic  system  was  designed 
and  implemented  at  the  University  of  California,  Irvine,  on 
an  IBM-PC-compatible  computer  with  a  66-MHz  Intel  486 
processor,  32  MB  of  random  access  memory,  and  a  17-inch 
CTX  monitor  with  1,280  x  1,024  pixels  running  on  a  Micro¬ 
soft  (Redmond,  Wash)  Windows  95  operating  system.  The 
software  development  environment  was  Microsoft  Visual 
C++  4.0  for  image-processing  functions  and  Microsoft  Vi¬ 
sual  Basic  5.0  for  the  user  interface.  Testing  of  this  system 
was  performed  at  KDMC  on  another  IBM-PC-compatible 
computer  with  a  233-MHz  Pentium  II  processor,  64  MB  of 
random  access  memory,  and  a  17-inch  monitor  (Vivitron; 
Gateway  2000,  Sioux  City,  SD)  with  1,280  x  1,024  pixels 
running  on  a  Microsoft  Windows  95  operating  system. 


RESULTS 


The  ROC  curves,  labeled  by  their  values,  are  shown 
in  Figures  4-8.  The  statistical  significance  of  the  changes 


False-Positive  Fraction 


Figure  6.  ROC  curves  for  radiologist  Rg  and  for  the  computer 
alone. 

in  these  values  is  reported  in  Table  2.  The  columns  of  A^ 
values  for  the  unaided  radiologist,  the  aided  radiologist, 
and  the  computer  alone  are  labeled  A,  B,  and  C,  respec¬ 
tively.  The  column  labeled  A-B  denotes  the  values  of  P  as¬ 
sociated  with  the  increase  in  obtained  by  replacing  an 
unaided  radiologist  with  the  corresponding  aided  radiolo¬ 
gist.  The  values  of  P  listed  in  Table  2  are  two-tailed  and 
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Figure  7.  ROC  curves  for  radiologist  and  for  the  computer 
alone. 


Table  2 

Comparison  of  ROC  Curves  for  the  Unaided  Radiologist,  the 
Aided  Radiologist,  and  the  Computer  Alone 


Reader 

P 

A,  Unaided 

B,  Aided 

C,  Computer 
Alone 

A-B, Two- 
tailed 

R, 

0.7150 

0.8354 

0.7533 

0.0157 

R, 

0.7179 

0.8814 

0.7533 

0.0013 

R3 

0.6277 

0.7538 

0.7533 

0.0015 

R4 

0.7157 

0.8077 

0.7533 

0.0636 

were  calculated  under  the  assumption  that  the  distribution  of 
each  estimated  change  in  A  ,  divided  by  the  estimated  stan¬ 
dard  error  of  that  change,  arose  from  a  normal  distribution. 
These  values  of  P  were  computed  by  using  the  ROCKIT 
software  (C.  E.  Metz,  ROCKEIT  0.9B,  beta  version.  Univer¬ 
sity  of  Chicago,  Chicago,  Ill,  1998). 

The  information  in  Figures  4—7  indicates  that  {a)  every 
aided  radiologist  outperformed  the  same  unaided  radiologist, 
{b)  no  unaided  radiologist  outperformed  the  computer  alone, 
and  (c)  three  aided  radiologists  outperformed  the  computer 
alone  and  the  fourth  aided  radiologist  (R3)  performed  ap¬ 
proximately  equal  to  the  computer  alone.  The  information 
in  Table  2  indicates  that  the  A^  values  of  the  unaided  radi¬ 
ologists  ranged  from  0.6277  to  0.7179,  and  that  those  of 
the  aided  radiologists  ranged  from  0.7538  to  0.8814.  The 
size  of  the  changes  in  A^  among  the  radiologists  ranged  from 


Figure  8.  ROC  curves  for  the  average  performance  of  the  four 
radiologists. 


0.0920  to  0.1635.  Table  2  also  indicates  that  the  A-B  changes 
in  A^  (ie,  unaided-to-aided  changes  in  A^  were  statistically 
significant  at  P  <  .002  for  radiologists  R^  and  R3,  at  P  <  .02 
for  radiologist  R^,  and  at  P  <  .07  for  radiologist  R^.  After  ap¬ 
plication  of  the  Bonferroni  correction  for  multiple  compari¬ 
sons,  these  three  P  bounds  became  .008,  .08,  and  .28,  respec¬ 
tively  (29).  Other  changes  in  A^ — namely,  A-C  (ie,  unaided- 
to-computer  alone  changes)  and  B-C  (ie,  aided-to-computer 
alone  changes) — were  not  statistically  significant. 

Figure  8  shows  the  average  performance  of  the  four  radi¬ 
ologists  when  unaided  by  the  mapped-database  system  and 
when  aided  by  the  mapped-database  system.  Results  of  the 
LABMRMC  analysis  indicate  that  performance  of  the  aided 
radiologists,  as  a  group,  was  statistically  superior  to  that  of 
the  unaided  radiologists  as  a  group  (P  <  .05). 

At  KDMC,  the  diagnostic  scores  S  and  M  are  convention¬ 
ally  interpreted  as  being  biopsy  recommended,  and  N,  B,  and 
P  are  interpreted  as  being  biopsy  not  recommended.  In  ac¬ 
cordance  with  this  convention,  we  calculated  the  numbers  of 
negative  biopsies  and  misdiagnosed  cancers,  as  well  as  the 
sensitivities  and  specificities  of  each  radiologist,  both  aided 
and  unaided  by  the  mapped-database  system.  These  results 
are  shown  in  Tables  3-5.  Tables  3  and  4  show  the  numbers 
of  recommended  benign  biopsies  and  misdiagnosed  cancers. 
Table  5  shows  the  sensitivities  and  specificities.  The  mapped- 
database  diagnostic  system  helped  every  radiologist  reduce 
the  number  of  benign  biopsies  by  40%  or  more  while  simul¬ 
taneously  reducing  the  number  of  misdiagnosed  cancers.  All 
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Table  3 

Numbers  of  Recommended  Benign  Biopsies  for  the  Unaided 
Radioiogist  and  the  Aided  Radiologist 


Reader 

Unaided  Radiologist 

Aided  Radiologist 

R, 

18 

11 

R2 

15 

8 

R3 

28 

18 

R4 

18 

10 

NN 

24 

Note— Number  of  ROls  from  benign  cases:  57.  NN  =  neural 
network. 
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Tabie  4 

Numbers  of  Misdiagnosed  Cancers  for  the  Unaided 
Radioiogist  and  the  Aided  Radioiogist 


Reader 

Unaided  Radiologist 

Aided  Radiologist 

R, 

11 

8 

R. 

12 

7 

R3 

7 

4 

R4 

8 

3 

NN 

4 

Note. — Number  of  ROls  from  malignant  cases:  23.  NN  =  neural 
network. 


the  radiologists  also  increased  both  their  sensitivities  and 
specificities  through  their  interaction  with  the  mapped-data- 
base  system.  The  statistical  significances  of  these  increases 
ranged  from  P  <  .01  to  P  <  .12  on  the  basis  of  a  comparison 
of  proportions  (30). 

As  mentioned,  the  sensitivities,  specificities,  and  ROCs 
were  calculated  on  the  basis  of  the  160  abnormal  ROls  and 
the  responses  of  the  panel  of  radiologists  to  them.  The  re¬ 
sponses  of  the  panel  of  radiologists  to  the  20  normal  ROls 
(the  remaining  20  normal  ROls  were  in  the  set  were 
excluded  from  these  calculations,  because  in  this  study,  the 
mapped-database  diagnostic  system  was  designed  only  to 
interpret  abnormal  ROls. 


DISCUSSION 


Figures  4-8  and  Tables  2-5  indicate  that  the  diagnostic 
performances  of  the  aided  radiologists  were  statistically 
significantly  improved  compared  with  those  of  the  unaided 
radiologists.  Table  2  shows  that  the  sizes  of  these  improve¬ 
ments  in  for  the  current  study  are  comparable  to  those  in 
found  during  a  recent  study  concerning  computer-aided 
characterization  of  manunographic  masses  (20). 

We  are  also  encouraged  by  the  improved  performance  of 
the  aided  radiologists  over  that  of  the  computer  alone,  as  in¬ 
dicated  in  Figures  4,  5,  7,  and  8  and  in  Table  2,  although  the 
number  of  cases  was  insufficient  to  prove  statistical  signifi¬ 
cance  of  these  improvements.  Table  5  shows  that  the  mapped- 
database  diagnostic  system  enabled  substantial  increases  in 
specificities  and  sensitivities  for  all  the  radiologists  in  our 
panel.  We  conclude  that  for  ROls  containing  microcalcifica¬ 
tions,  the  mapped-database  diagnostic  system  shows  promise 
in  helping  most  radiologists  to  raise  their  diagnostic  perfor¬ 
mances  substantially  over  their  unaided  performances  while 
providing  relevant  images  from  a  proved  database  to  support 
the  radiologists’  aided  diagnoses.  In  particular,  these  radiolo¬ 


gists  may  achieve  substantial  reductions  in  the  number  of 
benign  biopsies  and  misdiagnosed  cancers. 

The  levels  of  postresidency  mammographic  experience 
among  the  radiologists  in  this  study  ranged  from  29  years 
to  6  months  (R^  >  R2  >  R3  >  R^).  Figures  4-7  suggest  that 
all  radiologists  are  likely  to  benefit  substantially  from  ac¬ 
cess  to  the  mapped-database  diagnostic  system  regardless 
of  their  experience.  This  is  important,  because  radiologists 
vary  widely  in  their  diagnostic  skills — even  among  those 
who  rate  themselves  as  “experts”  (31).  Whether  the  size  of 
the  benefit  correlates  with  the  amount  of  experience,  how¬ 
ever,  is  not  clear  from  Figures  4-7.  (It  is  interesting  that  the 
performance  of  the  aided  R2  exceeded  that  of  the  aided  R^, 
even  though  the  experience  of  R^  was  greater.) 

The  results  of  this  study  suggest  that  a  properly  designed 
interface  between  a  human  reader  and  a  proved  database  of 
mammographic  images  may  enable  less  experienced  readers 
to  exceed  the  performances  of  unaided,  highly  skilled  read¬ 
ers — without  the  need  for  time-consuming  training  on  fea¬ 
ture  extraction  as  exemplified  by  Getty  et  al  (19).  Further¬ 
more,  our  diagnostic  system  represents  an  improvement  over 
earlier  systems  (17,19)  by  automating  the  selection  of  visual 
features. 

This  study  was  limited,  however,  by  the  quality  of  the 
monitors  used  for  viewing  the  images,  the  size  and  compre¬ 
hensiveness  of  the  mammographic  database,  the  lack  of 
full-field  and  earlier  views,  the  restriction  of  the  lesions  to 
microcalcifications,  and  a  possible  “reading-order  effect” 
associated  with  the  close  succession  of  unaided  and  aided 
diagnoses  (32).  Our  next  investigation  of  the  mapped-data¬ 
base  diagnostic  system  will  include  several  enhancements 
to  overcome  these  limitations.  The  images  will  be  viewed 
on  state-of-the-art,  high-resolution  radiographic  monitors, 
and  the  number  of  cases  will  be  increased.  The  tested  im¬ 
ages  will  also  include  full-field  craniocaudal  and  mediolat- 
eral  oblique  or  lateral  views  of  each  breast,  and  the  lesions 
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Table  5 

Statistical  Significance  of  Changes  in  Performance 


Reader 

Unaided  Radiologist  (%) 

Aided  Radiologik  {%) 

Change  in  Sensitivity 

Change  In  Specificity 

Change  (%)  P 

Sensitivity 

Specificity 

Sensitivity 

Specificity 

Change  (%) 

P 

R, 

47.8 

73.7 

69.6 

86.0 

+21.8 

<.01 

+12.3 

.12 

R, 

65.2 

68.4 

87.0 

82.5 

+21.8 

<.01 

+14.1 

<.05 

R3 

52.2 

68.4 

65.2 

80.7 

+13.0 

.09 

+12.3 

.07 

R4 

69.6 

50.9 

82.6 

68.4 

+13.0 

<.05 

+17.5 

<.05 

will  include  both  masses  and  microcalcifications.  In  addi¬ 
tion,  the  aided  and  unaided  diagnoses  will  be  separated  by 
at  least  1  month. 
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